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SUMMARY AND DISCUSSION 



The Manukau Family Literacy Project (MFLP) has now completed two full years 
of implementation since its inception. This report presents the findings of a 
summative evaluation for 2004, which includes a review of the outcomes 
achieved by the participants and also discusses some broader issues relating to 
further development of the MFLP model. 



Programme description 

Each MFLP programme has three partners (an early childhood centre (ECE), a 
school and a tertiary provider). During the course of this pilot the partners for the 
Bairds Otara programme were Bairds Kindergarten, Bairds Mainfreight School 
and Manukau Institute of Technology. The Rowandale partnership involved 
Manurewa West Independent Kindergarten, Rowandale School and Auckland 
University of Technology (AUT). 1 

The programme has four components (adult literacy, child literacy, Parent and 
Child Together Time (PACTT) and Parent Education). Adult participants take 
part in a full-time tertiary programme on a school site; they work with one of their 
children in literacy-related activities during daily PACTT time; the Parent 
Education component allows them to observe and study child development and 
behaviour as part of their adult education course; and parent, child and wider 
whanau also take part in regular literacy-related events and activities. Integration 
of the components to maximise the learning outcomes for all participants is a 
key aspect of the programme. 



Description of participants 

Over the time covered by this report, there have been five intakes of participants 
- 70 adults and 70 nominated children, split roughly between the two years. 

> MFLP adult participants were overwhelmingly female, with an average 
age of 33. 

> On average, participants had three children; the nominated children in 
the programme had an average age of five. 

> Two-thirds of adult participants were Pasifika and almost all the rest 
Maori. 

> Seventy percent of the adults had no school qualifications and just under 
half had no post-school qualifications. 



1 Dawson Road did not start until later in 2004, which did not leave enough time for 
substantive outcome data. Therefore, only a minimal amount of data on that programme 
has been included in this evaluation. 
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> Most had worked in unskilled or semi-skilled work and a quarter of them 
were not in paid work prior to the course. 



Programme outcomes 

As part of MFLP’s initial development, those involved in the Bairds Otara and 
Rowandale programmes devised a set of goals for their sites at a professional 
development day in 2003 (see Appendix A for detailed lists). Essentially, the 
goals crystallise into the following areas: 

> Foundation skill gains for both adults and children 

> Build parents’ levels of self-confidence and self-efficacy 

> Raise and identify long-term aims for education and employment 

> Encourage parents’ involvement in their children’s education 

> Explore and adopt new parenting skills 

> Build learning communities among parents, their children and 
participating institutions. 

These goals have not been revised since that time and have therefore been 
taken as the broad guideline for this evaluation. 



Foundation skills 



The great majority of the adults who have enrolled in the MFLP have had 
minimal success in the schooling system (as evidenced in their very low 
numbers of qualifications) as well as minimal educational experience since 
leaving school. In the course of their MFLP experience, most of the adult 
participants have successfully completed tertiary courses, some with notable 
pass levels. 



In terms of their self-assessments, where they compared their foundation skills 
at the beginning of the year with the end, a clear majority of the students 
reported that they had made gains in their reading, writing, spelling, numeracy, 
computing and library skills. Many students perceived that they had made gains 
in all of these areas (although some still had a ‘spiky’ skill profile) and only one 
student consistently self-assessed as having made little or no progress in her 
foundation skills. Qualitative data from interviews confirmed that the students felt 
much more positive about their skills and how to use them in their study. 

While some children did make considerable gains in their reading and writing, 
the data is not as consistent as that of the parents. Feedback from both teachers 
and parents show that one group of children had improved noticeably during the 
year and this gain was attributed at least in part to the MFLP. Other students 
had been making good progress prior to becoming involved in the programme 
and this progress had continued. A smaller group of only a few children were felt 
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to have made little or no progress. The assessments of the children’s 
before/after Running Records and Maths Strategy levels did show some positive 
gains for the MFLP children in comparison with their non-MFLP controls, but the 
differences were not great and the findings need to be treated with some caution 
because of methodological issues in these measures (the small sample 
numbers, control selection). 



The uneven pattern of gain among family literacy children has also been 
reported in evaluations of family literacy children in the US (Paratore, Melzi, & 
Krol-Sinclair, 1999). 



Self-confidence and self-efficacy 



Gains in self-confidence and self-efficacy are probably one of the most notable 
outcomes of this programme, both for the adults and the children involved. 
There is consistent evidence from a range of sources that family literacy 
participants feel significantly more confident in a range of contexts, from talking 
to friends through to speaking to large groups. In more general terms, the 
parents now feel much more confident about making plans for their futures, 
partly because they have been able to explore a range of options during the 
course, but mostly because they believe in their own abilities and skills to work 
towards these goals. 

Allied to their greater levels of self-confidence, the adults’ self-efficacy levels 
have clearly risen. Most of the adults in the MFLP have been out of school for 
many years and did not succeed much as students at school. They come into 
the MFLP with low levels of confidence about their ability to use the skills they 
already have, especially in some subjects such as maths. By building their self- 
efficacy (in which the programme is clearly very successful), the learners 
become immersed in a positive spiral where self-efficacy leads to improved skill 
levels, which in turn leads to greater self-efficacy. 



Long-term education and work aims 

The great majority of MFLP enrolees have been either out of paid employment 
or in low-skill jobs previously; similarly, most have had no, or very limited, 
previous tertiary education. As a result of the MFLP, most now aspire to 
undertaking further study or moving into more skilled jobs. Very few are 
interested in returning to what they were doing previously. 

This report has added a very useful dimension to the evaluation by tracking 
MFLP participants who have been out of the programme for over a year. The 
data from these participants adds credibility to the aims and plans of those just 
completing the programme. Data from the 2003 groups shows that a significant 
number of these adults have indeed gone on to do what they intended, or 
something similar, at the end of that year. Of those contacted, very few were 
doing what they did prior to the MFLP and even in the cases where they were 
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not in paid employment or study, the adults were fulfilling valuable social roles 
for their whanau/community to a greater extent than before the course. 

This evaluation has not been able to report much about what has happened to 
those students who withdrew from both the 2003 and the 2004 programmes, as 
most of these people were not readily contactable. These students withdrew for 
a number of reasons (not always of their own making and often for positive 
reasons) and possibly warrant special attention in any future evaluation of the 
programme. 

Parental involvement in children’s education 



Both of the schools whose participants were covered in this evaluation have long 
struggled to involve parents beyond peripheral activities where they feel 
comfortable, such as festivals and fund-raising; the kindergartens struggle to 
involve parents who ‘drop ‘n go’. Figure 1 illustrates the range of ways parents 
are involved in schools and early childhood programmes. 



There was considerable feedback from a number of sources that the family 
literacy programmes have helped change these situations. MFLP students have 
stood for Board of Trustee elections (necessitating ballots where there had often 
not been sufficient candidates for one), they (and their whanau) are a constant 
physical presence in the schools/kindergartens for most of the week, they are 
more confident about approaching teachers and they are active participants in 
their children’s classrooms during PACTT. At home, family literacy parents 
report working with their children on their homework in a more involved way and 
for the first time in many cases. They are able to do this because of their own 
self-efficacy and newly-acquired academic skills. Most importantly, they value 
this new role and see its significance in their children’s long-term educational 
development. 



One of the distinctive features of the MFLP is that the adult participants are 
involved for sustained period (30 weeks) and receive intensive teaching 
throughout this period - both of which are considered conditions for learner 
gains in the international research literature (Benseman, Sutton, & Lander, 
2005). 



All of these points indicate that an outcome of family literacy is the increased 
involvement of parents in their children’s education, as summarised in the 
diagram below. It is interesting to note that three of the 2003 participants have 
been working as teacher aides in one of the schools, a role that has enabled 
them to continue contact with the programme, on-going contact with the school 
and also clarify career goals (one person has changed from wanting to be an 
early childhood teacher to a primary teacher as a result of working in the 
school). 
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Figure 1. Parental involvement in schools and early childhood centres 




Parenting skills 



Parenting is a cornerstone of family literacy and a key element of the MFLP 
because parents play a vital role in enhancing the achievement of their children. 
There is evidence that the academic and social development of children can be 
enhanced when there are strong partnerships between home and school/EC 
and when parents participate in adult education that helps them work 
constructively with their children (Biddulph, Biddulph, & Biddulph, 2003). 
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The exploration and debating of parenting issues and skills emerges in a 
number of ways in the programmes, ranging from formal course content through 
to informal discussion and peer support. Some family literacy programmes have 
been criticised as prescriptive and based on a deficit approach (Auerbach, 
1994), but the parenting component of the MFLP shows no evidence of these 
attributes. Parents have been exposed to information and challenges about 
parenting, but have engaged with this content in an open and critical way, which 
is in keeping with the stated philosophy of MFLP to positively engage parents in 
this area. 



As a result of this engagement, there are clear signs of participants wanting to 
do things differently as parents and feedback that they are acting in new ways in 
this role. Feedback indicates that they have gained new energy and commitment 
as parents fed by new information (such as child development) and alternative 
models of doing things in areas such as discipline. They are more interested in 
what their children are doing at school and more able to help them with their 
homework because of their own progress as learners. 



Another outcome of the programme is the role modelling of different 
expectations and ambitions for children. Because the programme has thus far 
been physically located on school sites with the participating early childhood 
centres also on-site or very close, children (including non-MFLP ones) see 
parents engaged in education every day in communities where few parents 
participate in post-school education. The MFLP families see their parents as 
active learners not only at school, but also at home - as epitomised by parents 
and children doing their homework together. 



Learning communities 

Traditionally our education system has been strongly stratified by age - layers of 
learners are defined by their age, with little mixing between them either by the 
learners themselves or the institutions that provide the programmes. Family 
literacy cuts across this age-stratification by mixing early childhood, primary 
schools and tertiary education. This improved integration has had positive 
effects for the providers, with increased awareness of each other’s work, 
improved co-ordination of programmes, an awareness of being part of a larger 
educational system and a more prominent profile in the broader community. For 
the parents, there is increased awareness of the overall educational system, 
while non-family literacy parents see alternative models in operation for people 
like themselves. 



Family literacy is a means of bringing parents into schools and early childhood 
centres in a more sustained way and thereby helping to de-mystify education. 
The adults see teachers (both in schools and early childhood centres) working 
with children and exposed to alternative teaching methods they can use 
themselves. Teachers on the other hand, have parents involved in their 
classrooms on a regular basis, providing opportunities to interact with them that 
rarely happens otherwise. 
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Broader issues 

Key stakeholders from the pilot were interviewed to find out more about how the 
model had been implemented. Respondents identified a number of roles 
undertaken by COMET during the pilot, including providing leadership and 
vision, guiding the alignment of the curriculum and the four components to get 
the most out of the programme, accessing funding, administration and support, 
providing crisis management and specialist expertise. 



Almost all the participants thought that an independent broker was necessary in 
cross-sector projects such as the pilot because none of the partners on their 
own had the time, expertise or knowledge of other parts of the education sector 
to be able to take on family literacy alone. COMET was able to take a wider view 
and challenged partners to see outside their own sphere of knowledge. 



One of the most complex issues during the pilot was related to identifying and 
then delivering the appropriate qualification for the adult education component in 
a manner that provided enough flexibility for the adults and also made it possible 
for the four components of the model to be aligned. Another related issue was 
childcare, particularly for children too young to attend the kindergarten 
programmes and for those whose parents went off-site for a proportion of their 
tertiary programme. In 2005 there will be only one tertiary provider for all MFLP 
programmes and all adult teaching will take place on site. Respondents also 
thought MFLP made heavy demands of time and meetings, but many 
anticipated these to diminish as the programme matured and that the benefits 
outweighed the work involved. 

Overall, those interviewed were very positive about the changes MFLP had 
brought about for the families involved and for the schools and early childhood 
centres. 



Conclusion 

The Ministry of Education’s Statement of Intent 2 lists a number of strategies that 
the government sees as essential to achieving its educational aims. They 
include: 

> raising expectations for achievements of all learners 

> strengthening family and community involvement 

> developing a collaborative and responsive education network. 



2 ( http://www.minedu.qovt.nz/index. cfm?lavout=document&documentid=9644&data=l) 
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In particular, the Ministry wants to build quality involvement with communities by 
focussing on five main areas: 

> strengthening families and whanau capability to support the learning of 
their children 

> strengthening families and whanau collaboration and engagement with 
providers and teachers 

> strengthening the engagement of families of children with special 
education needs in services and services development 

> encouraging community engagement to support parents, whanau and 
providers to raise student achievement 

> integrating education perspectives into wider social policies that 
contribute to raising student achievement. 



This evaluation report has shown that the MFLP has made a strong contribution 
towards these policy priorities. 



Firstly, the programme has been successful in recruiting (and retaining a high 
proportion) of adult learners who have historically been under-represented not 
only in the participation statistics of New Zealand education, but also the 
statistics of success in the educational system (Cain Johnson & Benseman, 
2005). These participants have typically left school early with few or no 
qualifications and worked in low status, low-skill occupations with limited 
aspirations for their own and their children’s futures. The great majority come 
into the programme as failures of the schooling system, and in some cases, the 
tertiary system, yet those who complete MFLP have about a 90% attendance 
record. 



Secondly, the MFLP has achieved a high rate of success in raising their 
academic skills. As an example, of the 23 grades achieved by students doing 
MIT study as part of MFLP, 20 were in the A range. The adults improved their 
self-confidence and self-efficacy, and also their long-term aspirations and 
ambitions. There is evidence that many who have been out of the programme 
for some time are also achieving these ambitions - they are doing what they 
said they intended to do. As parents, MFLP learners have become more 
involved in their schools and more active in their children’s education, both at 
school and at home. They are modelling new possibilities and provide valuable 
input for their children. There is some limited evidence that their children are 
performing better academically at school and are more confident and active 
socially than previously. 



Only one Rowandale participant had been in paid work and none in tertiary 
education prior to enrolling on the MFLP. A year on from the programme: 
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> at least eight were in some form of employment 

> two were studying 

> and six are planning on doing a tertiary course in the next few years. 

With the Bairds Otara groups, three were in paid employment and none was in 
tertiary education prior to enrolling in the MFLP. A year after the programme: 

> at least seven were in paid employment 3 

> six were in a tertiary programme 

> and two are planning on doing a tertiary course in the next year. 

While many have succeeded as a result of participating in this family literacy 
programme, it does not always achieve the same level of outcomes for all those 
who enrol. There have been withdrawals and some have not changed much 
either academically or in broader terms as people as parents, but most have. 
There is universal support for the programme and criticisms are limited to 
operational details. 



Thirdly, the MFLP is having effects beyond the learners themselves. The MFLP 
is contributing to a more integrated community of educational providers where it 
operates and it is valued by the project’s early childhood, primary and tertiary 
professionals for this outcome. The programme models a positive example of 
lifelong learning in action for the adults and children in the programme, for those 
professionals involved in the programme and increasingly, for children and 
parents not directly involved in the programme. 



As one key informant said, “This really is lifelong learning.” 



A final word goes to two of the students who made the following speeches at 
their graduation ceremony at the end of the year. 



3 A smaller percentage of the Bairds Otara students were able to be contacted, so these 
numbers probably under-report these outcomes. 
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This year has been a great year for me; I’ve met a whole new group of friends 
who are an awesome bunch. One reason for me being here tonight which I think 
is really special for me is my Dad. I lost my Dad two years ago and it was him 
that inspired me to get off my ass and do something with my life. He said ‘it’s 
time for you to get out there and do something for yourself and I thank him so 
very much for these words. I know he’d be very proud of me tonight, but my 
beautiful Mum is here to celebrate. 



On the first day of March 2004, with much trepidation and hesitation, 13 adults 
entered a classroom in this school with the number 5 on it. This was to become 
our home over the ten months that followed. When we embarked on this 
journey, we were all unaware of the impact this course was going to have on 
both ours and our families’ lives. No one person in this group had any concrete 
directions whatsoever. 

We all leave here tonight with direction and determination to pursue our future 
goals. We have waded through waters of all kinds - shallow, deep, muddy, clear 
and even shark-infested. But through all this, one thing remained clear. We 
became a tight-knit group who began to genuinely care for one another, almost 
as though we were all blood-related. . Through thick and through thin, eight of us 
remained and persevered and it is with so much pride that we all stand here 
tonight, this night, our graduation! Kia kaha whanau! 



Recommendations 



1. That MFLP partners establish multi-site goals and directions for 
2005-2006 

2. That COMET run a workshop for partners and stakeholders to review 
MFLP to date, based on the findings of this evaluation, with a view to: 

> refining the current model and programmes 

> exploring variations of the model that would be appropriate for 
expanding this form of family literacy in other contexts. 

3. That all sites collaborate to set up standard processes for collecting 
data. In particular, it would be useful to have some agreed 
standardised assessment processes across all sites (while accepting 
that particular sites may want to gather additional data for local 
purposes). 
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INTRODUCTION 



This is the fourth evaluation report on the pilot Manukau Family Literacy 
Programme (MFLP). The earlier reports were formative and process 
evaluations, documenting the development of the programme. 

> The first report (Benseman, 2002) provided an overview of family literacy 
programmes, a literature review of the research on these programmes 
and a analysis of the development of the Manukau Family Literacy 
Programme in its planning phase during 2002, with a particular emphasis 
on management and structural issues that arose in this initial phase of the 
project. 

> The second report (Benseman, 2003) reviewed the operations of the 
MFLP on the two pilot sites over their first six months of operations in 
2003, as well as some data on the impact of the programme on the first 
two intakes of participants and the issues that arose during that period. 

> The third report (Benseman, 2004b) covered the period of July to 
November 2003, when the first intake of participants at the Rowandale 
site in Manurewa completed their course and the second intake of 
participants at Bairds Mainfreight in Otara started and completed their 
course. 4 

This report has two parts: 

> A summative evaluation of the 2004 programme (Sections 1 - 5) which 
looks at programme outcomes of the MFLP, using a range of data 
sources for evidence of changes in both the adult participants and their 
children across a number of aspects - academic, family, personal and 
social. 

> A discussion of broader issues of MFLP (Sections 6-10) such as the 
nature and significance of the role COMET has played in establishing a 
family literacy programme, some of the challenges and issues that MFLP 
has had to deal with and the way the model of family literacy has 
developed since the pilot began. 



For readers who have not read the earlier reports, the next few pages outline the 
nature of the program and provide an overview of the MFLP’s operations. 
Readers familiar with this information could move on to the description of the 
summative evaluation process on page 19. 



4 For a summary of all three reports, see the Ministry of Education publication “I’m a 
different person now” (Benseman, 2004a). 
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Description of Manukau Family Literacy Programme 



The MFLP grew out of an initiative by the Literacy Taskforce of the City of 
Manukau Education Trust (COMET). The initial planning for the programmes 
was undertaken during 2002, culminating in two pilot sites starting operation in 
2003 at Bairds Otara and Rowandale in Manurewa; a third site at Dawson Road 
Primary School (known as Dawson Road Community) was added in late 2004. 

The MFLP has four components 5 : 

1. An adult education component designed to extend basic education skills, 
including teaching adult participants to: think critically and creatively, solve 
problems, set goals and achieve them and acquire successful interpersonal 
skills. 

2. Children’s education to promote the growth and development of young 
children and to engage parents in their child’s educational programme in 
order to foster meaningful involvement that will be maintained throughout the 
child’s educational career. 

3. Parent and Child Together Time (PACTT); in the MFLP PACTT has 
developed into three forms - Tahi PACTT (1:1), Classroom PACTT and 
Whanau PACTT. During Tahi PACTT, the parent/caregiver typically spends 
15-20 minutes per day with their nominated child, observing, playing and 
helping with their learning. 6 Classroom (Roopu) PACTT involves all PACTT 
children and participating parents in a shared literacy experience once a 
month. Whanau PACTT happens once a term and is an extended family 
experience usually out of school time. 7 Activities have included a family quiz 
and story telling in story and drama and the adult participants do much of the 
organisation. 

4. Parent time to provide instruction on how children grow, develop and learn to 
read and write, address issues critical to family well-being and success, 
connect parents with a wide array of community resources and provide 
parents with opportunities to network and develop mutual support systems 
with others in the programme. 



These four elements (which have been adapted from the Kenan model of Family 
Literacy that has been established in the US) are shown in the diagram below. 
Figure 1 also illustrates what is sometimes referred to as the fifth element of 
family literacy programmes - integration. As Potts (No date, p. 4) says 



5 These four components have evolved from the Kenan model of family literacy 
developed in the USA (see Figure 2). 

6 Tahi PACTT provides approximately 38 hours per year planned and focused attention 
by a parent on their child’s learning activities. This compares favourably to the estimate 
of 13.3 hours per year of explicit, planned and focused individual time a school student 
might receive from a teacher (Davis, 2002). 

7 Classroom PACTT equates to approximately five hours additional literacy-focused 
activity and Whanau PACTT approximately 15 hours per year. 
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Integration has become a critical, defining characteristic of family literacy 
services, working to create a system for delivery of curriculum, instruction 
and assessment. Rather than providing stand-alone, isolated services 
such as early childhood education or adult basic skills education, family 
literacy programmes bring parents and children together to learn, weaving 
key strategies and message throughout the four primary components.... 
Integration of these components is used intentionally as a cohesive 
system to promote learning within the family unit. 

See Potts (2004) for a fuller discussion of this element. 



Figure 2. Kenan model of family literacy's four components 




Each MFLP site involves three partner institutions - an early childhood 
centre/kindergarten, a primary school and a tertiary provider. 8 The kindergartens 
and the primary schools work with the child participants (one per parent) 
enrolled in the programme and link with the adult components for key parts of 



Most of the participating agencies have developed Memoranda of Understanding with 
COMET. 
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the programme such as Parents and Child Time Together (PACTT). The tertiary 
providers employ the adult educator who is responsible for teaching the adult 
participants, as well as some involvement in other components of the 
programme. All three MFLP programmes thus far have been located on primary 
school premises - Bairds Mainfreight, Rowandale Primary and Dawson Road 
Primary. 

The overall management structure of the MFLP is summarised in the following 
diagram provided by COMET . 9 



9 For a fuller account of the development of family literacy generally, see Wasik and 
Hermann (2004) and for additional details of the early stages of the MFLP see the first 
evaluation report (Benseman, 2002). 
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Figure 3. Manukau Family Literacy Programme Management structure 
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Description of summative evaluation project 



The purpose of this summative evaluation study is to report on the impact that 
the MFLP has had on its adult and child participants. The impact covers both the 
educational and social (especially in relation to parenting) outcomes; it covers 
not only the participants from the 2004 programme, but also a follow-up of those 
who participated in the programme in 2003. 

This evaluation has been funded by three government agencies: the Ministry of 
Education, the Tertiary Education Commission and the Ministry of Social 
Development. The involvement of the three agencies reflects the diversity of 
outcomes that have been identified as a result of family literacy. The evaluation 
results however have been written as an integrated report, covering all aspects 
of the programme’s impacts. 

Although the MFLP programmes at Bairds Otara and Rowandale started early in 
2004, the evaluation contract could not be finalised until September. The delay 
in finalising the contract was not ideal in terms of data collection for the 
evaluation (mainly in relation to the early childhood children), but the co- 
operation of key people in the project have minimised the ensuing difficulties. 

Most of the evaluation relates to the programmes at the Rowandale and Bairds 
Otara sites. Only a limited amount of the data from the Dawson Road 
Community site is included because this programme site did not start until late in 
2004. The socio-demographic characteristics of their adult learners have been 
included, but none of pre-testing data collected from the adult participants, as 
the programme was not of sufficient duration that any noticeable impact could be 
expected or measured. The pre-entry data collected in October from this site 
could be used however in any future evaluations. 



Methodology 

This evaluation drew on a wide range of sources involving both quantitative and 
qualitative data. Data sources included records of: 

> adult students’ work in MFLP (learning journals and other writing) 

> MFLP students’ tertiary academic achievements from official results 

> children’s school academic achievements 

> attendance (adults and children) 

> MFLP documentation and reports. 

There were also interviews (mainly face-to-face, but also some by phone) with: 

> 2004 MFLP adult participants 
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> adult participants from the 2003 programme 

> adult educators from the tertiary institutions 

> early childhood teachers 

> teachers in participating schools 

> other key informants linked to the programme (e.g. school-based social 
worker, COMET staff). 

Where appropriate, further details of the methodology are included with the 
following sub-sections of results. 



Ethics 



The main ethical issues in this evaluation involved maintaining the confidentiality 
of the respondents in the study and gaining informed consent from all those 
involved. Ethics approval for the project was given by the Human Participants 
Ethics Committee of The University of Auckland (reference 2004/242). The 
equivalent ethics committees of Manukau Institute of Technology and Auckland 
University of Technology also granted ethics approval as the students are 
enrolled with and the teachers in the programmes employed by these 
institutions. All of the 2003 MFLP participants had already signed Consent 
Forms in 2004, which covered a period of three years. The adult participants 
gave consent for access to their nominated children’s school records. 

No-one refused to participate in the evaluation. 
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SUMMATIVE EVALUATION 



1 THE 2004 MFLP PARTICIPANTS 



A total of 66 adults and nominated children participated in the MFLP during 2004 
(Table 1 below). 



Table 1 . Summary data on 2004 MFLP participants 





Bairds Otara 


Rowandale 


Dawson Rd 
Community* 


TOTAL 


Total adult 
participants 


10 


14 


9 


33 


Total nominated 
children 


10 


14 


9 


33 


Pre-schoolers 


4 


7 


3 


14 


Primary school 
children 


6 


7 


6 


19 


Withdrawals 


1 


3 


1 


5 


Men 


1 


1 


1 


3 


No. of days in 2004 


79 


115 


31 


225 


Average attendance 
all enrolments 


85.9% 


68.2% 


NA 


77% 


Average attendance 
of completers 


91 .8% 


89.1% 


NA 


90.5% 


Number in paid 
employment prior to 
enrolling in MFLP 


3 


3 


2 


8 



* data not supplied because of short duration of programme 

As the table above shows, each of the sites had one male participant. 10 This 
pattern mirrors the gender distribution of 2003, where each of those intakes also 
had a single male participant (one had two males, but one withdrew) and also 
overseas experience where family literacy participants are overwhelmingly 
female (Padak, Sapin, & Baycich, 2002). The average ages were 31 years at 
Rowandale (ranging from 18 to 40 years) and 34 years at Bairds Otara (ranging 
from 23 to 43). Two of the Rowandale adult participants were a mother and 
daughter and one adult at Dawson Road had her niece as her nominated child; 
these patterns continue those of 2003 where a number of participants were 



10 In order to preserve the anonymity of the two male participants, all the adult 
participants are referred to as she. 
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grand-parents or wider family members. Participants had an average of 3.3 
children and the average age of the nominated children in the 2004 MFLP was 
5.4 years. 



The ethnicity of the 2004 participants across all three sites is shown in the 
following graphs. Two-thirds of the participants are Pasifika, of whom most are 
Samoan or Cook Islanders. Each of the programme sites is distinctive in terms 
of their participants’ ethnicities: Dawson Road Community is predominantly 
Samoan, a majority of Bairds Otara participants are Cook Island and most of the 
Rowandale site are Maori. There is only one Pakeha participant in the MFLP. 
The high numbers of Pasifika and Maori participants is reflective both of their 
catchment areas generally and the participating schools specifically. 

Figure 4. Ethnicity of MFLP adult participants, 2004 (n= 33) 



Pakeha 

3 . 2 % 



Maori 

29 . 0 % 




Pasifika 

67 . 7 % 



Cook Is 
40 . 0 % 



Samoan 

44 . 0 % 




Tongan 

8 . 0 % 



Niue/Cook Is 
8 . 0 % 



A third (1 1 ) of the participants said that their first language was not English; nine 
said it was Samoan; one was Niuean and one Tongan. Nineteen had been born 
outside New Zealand (all in Pacific Island states), but had averaged 18 years 
residency in New Zealand. Fourteen said that English was not the main 
language spoken in their home; eight said it was Samoan, three Cook Island 
Maori and one each for Niuean, Tongan and Maori. 



The following graph summarises the participants’ highest school qualifications. 
Seventy per cent had no qualifications at all. 
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Figure 5. MFLP adults’ highest school qualifications (n=33) 



6th Form Cert. 
13 . 3 % 




70 . 0 % 



The next graph summarises any qualifications the participants have gained 
since leaving school. Nearly half have not done any education since leaving 
school; those who have completed a qualification are all at low levels. 

Figure 6. MFLP adults’ post-school qualifications (n=33) 




None 

46 . 9 % 



In terms of their most recent work history, a quarter (8) had not been in paid 
employment prior to starting the MFLP course and had been working at home 
and/or on the Domestic Purposes Benefit. Most of the remaining 25 had worked 
in un- and semi-skilled jobs such as kitchen-hands, takeaway bars, factories, 
cleaning and shops. A small number had worked in skilled jobs such as 
machinists, panel-beater and call centres. 

Teachers at the early childhood centres or the primary schools remained by far 
the most frequent source of information about the MFLP and how they were 
recruited, although some also mentioned previous MFLP participants and their 
children. 
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2 GENERAL FEEDBACK ABOUTTHE MFLP 



Interviews were held with the participating schools’ principals, the adult 
educators, those teachers who had nominated children in their classes (primary 
and early childhood), the adult participants and some additional key informants 
such as a school social worker about their evaluations of the MFLP. 



All of these interviewees were very positive about the programme from their 
perspectives. 



2.1 School principals' perspectives 



Both of the school principals interviewed were in the second year of MFLP 
operation and agreed that the programme was now running more smoothly than 
in 2003. Even though some of the staff at both sites had changed and these 
changes had necessitated familiarising the new people with the programme’s 
operations and philosophy, the routines and structures established in the first 
year meant that operationally the programmes ran on a more routine basis 
requiring slightly less input from the schools and kindergartens - “family literacy 
is part of how we do it now and it’s no longer onerous.” 



Both principals were extremely supportive of the MFLP in terms of its value for 
the children involved, their parents and the broader community. They were also 
clear that the programme is no universal panacea - a 2003 participant in one 
school for example had been recently ‘chased up’ over her child’s attendance 
irregularities during 2004. On the other hand, some of the difficulties associated 
with the participants were considerable and take time to resolve successfully - 
one principal gave the example of an MFLP parent who had withdrawn from the 
programme in order to escape from a partner manufacturing and using P; she 
had since moved to another area with a new partner in a much more positive 
frame of mind (she said “he really looks after me”) and was now looking to 
regain the custody of her children from her mother because they were “roaming 
the streets and getting into trouble.” This principal was optimistic that the parent 
had been able to turn her life around successfully after a period of real difficulty. 
The case also illustrates that withdrawal from a programme is not necessarily a 
negative outcome for the learner. 



The principals were adamant that the programme had had a broader impact on 
their schools than just the parents and their nominated children participating in 
the programme. Both principals said that historically their schools had really 
struggled to get parents involved in their programmes and day-to-day activities, 
despite constant efforts to do so. As one principal said: 



Real parental involvement in this school has been zilch. We have tried 
everything - reading mornings, maths mornings, free computer courses - 
some of these worked at first, but nothing really worked. They turn up for 
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festivals and so on, but you can’t get them involved in planning or 
curriculum sessions. But family literacy is changing that and it’s become 
part of the culture of the school. 

The other principal says that as a result of MFLP she now knows not only the 
parents really well, but also their extended families because “they’re in and out 
of the school all the time.” This school had also been using former MFLP parents 
to gather feedback from other parents on a range of issues after finding 
questionnaires totally ineffective. 

At one school three of the 2003 MFLP participants had stood for the Board of 
Trustees election, 11 several were now working as teacher aides in the school 
and the parents were much more prominent in the school and more confident in 
approaching teachers about issues. This principal felt that the effects of MFLP 
had spread well beyond the 28 parents who had participated in the programme 
to date. 

It’s helped raise the value of education around here hugely - people just 
don’t see education as important in this community, but family literacy has 
opened their eyes to this. People hear about it even if they aren’t in the 
programme. 

This principal felt that one of the key features of the programme was that 
because the parents are on-site for long periods of time they understood the way 
the school functioned better than parents who only spend short times at the 
school, and then usually at times such as before and after school when there is 
little happening educationally: 



They [MFLP parents] see the raw operations of the school, the good and 
the bad and so they understand the challenges of the job. 

Both principals said that because MFLP children were in both early childhood 
programmes and at school, this had led to closer relationships with the 
participating kindergartens than they had had previously - even though both 
schools are in close physical proximity. Staff often came to each other’s social 
events and the kindergarten children attended school celebrations and 
occasional school assemblies. 



Both principals reported that while some of the earlier intakes had recruited from 
families where there had been relatively few problems, the MFLP had managed 
to recruit “some of the more difficult families” in subsequent intakes. They were 
satisfied that the recruitment was now successful in targeting high need families 
with poor attendance records, high academic and social needs (including social 
worker and CYPS involvement) and those who had had minimal involvement in 
the school previously. 



11 One family literacy parent stood unsuccessfully at the other site. 
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With regard to their assessments ot the impact of MFLP nominated children, 
both the principals were cautiously optimistic. One said: 

There’s not been a huge academic gain as yet, but you can see big 
changes in their [children’s] self-confidence and it’s essential [for them] to 
ask for help and say ‘I don’t understand’. The parents model how it can 
work (it’s wonderful seeing all these parents going off to university and 
these courses). So if Mum’s here [at school], then ‘I can do it’ - so kids 
ask more questions and that’s essential. 

One principal talked about one of the 2003 participants who started her teacher 
training in 2004; this student told the principal that her goal is to come back to 
the school as a teacher - “I owe you so much.” This school’s commitment to 
family literacy is reflected in the fact that they are currently planning an 
extension to their hall and will incorporate a family literacy room into it. 

The greatest issues from the principals’ perspectives were: 

> recruiting sufficient numbers of adult participants 

> less involvement of parents on-site at Bairds Otara in 2004 because 
parents were off-site much more studying at MIT (also less whanau 
involvement because of this factor) 

> arranging childcare before and after school for MFLP children 

> not being able to recruit parents with children aged 0-3 years. 

Overall however, both principals were adamant that family literacy was a very 
positive development - as one principal said, “it’s been a really positive move for 
this school.” She then listed why: 

> it gives the school a sense of community by bringing parents in a on a 
sustained basis 

> it models lifelong learning for the children - “they know their teachers do 
PD, but now they see their parents and the message they get is ‘if my 
parents think it’s important enough to be involved, then it should be 
important for me too” 

> they see that education is important, it’s part of life, “so we get a culture 
of a learning school, a learning community” 

> staff feel that they are part of a broader education community with 
multiple providers on-site 

> it has given the school a profile 

> teachers are more open to parents coming into their classrooms 

> “it’s habit-forming for the parents - because of the sustained duration of 
the programme, they get into positive routines and habits.” 
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2.2 Adult educators' perspectives 



Both of the adult educators interviewed were also positive about what had been 
achieved in their programmes over the year. One of the sites had experienced 
difficulties with personality clashes between some of the students, which 
persisted throughout most of the year. While this issue resulted in some 
‘personal dramas’ at times, there was sufficient resolution to ensure that the 
group functioned satisfactorily through to the conclusion of the programme. This 
site also had problems with some students’ attendance at times, requiring 
considerable work for the adult educator to avoid further withdrawals. 

The educators were asked what they felt they thought the key aspects of the 
programme were in helping to achieve the outcomes. 



It’s a combination of everything really, the practical and the theoretical, 
how it’s related to their lives (especially the exercises they do with their 
children), the importance of relationships and the sustained period of 
learning with one teacher. 



The Rowandale teacher felt that her professional isolation had improved from 
the previous year and her study for an M Ed degree had been very beneficial for 
the opportunities it offered to focus on family literacy research and literature. The 
relationships with the participating school and kindergarten had further 
strengthened and were now a real strength of the site. 



Rather than being taught mainly on-site as an integrated whole (as happened in 
2003), the Bairds Otara students in 2004 were enrolled in an array of Manukau 
Institute of Technology courses taught on the polytechnic site, while the Human 
Development course was taught at the school. This strategy was done in order 
to try and match the range of students’ literacy skills to appropriate course 
levels, but received a mixed reaction from programme participants and project 
personnel. Some people felt that the reduced physical presence at the school 
reduced the amount of interaction between the parents and the school and 
reduced the visibility of the programme generally. On the other hand, others felt 
that the range of course better matched the students’ learning needs and helped 
increase the tertiary education profile in the programme. 

Issues raised by the two adult educators included: 

> poor communication at times about the co-ordination of programmes 
between the Bairds Otara schools, kindergartens and the tertiary partner 

> confusion from having multiple partners in the programme - “who do you 
listen to?” 
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2.3 Primary and early childhood teachers' perspectives 



There was a wide range of experiences and assessments in relation to MFLP 
among the teachers of the nominated children. However even when they had 
seen little progress in the nominated children’s academic skills, the teachers 
were very positive and supportive of the MFLP because they saw benefits for 
the children in non-academic areas, they valued the way that parents were able 
to model learning behaviour for their children (“it’s great for showing that learning 
never stops”) and they appreciated the extensive contact that they had 
experienced with the parents involved. 



You interact a lot more with these parents than with the ‘drop-‘n-go’ ones. 

MFLP is just brilliant, it’s so good for the community and the children. The 
relationships with the parents have just grown, parents coming in for 
morning tea, so the news just spreads out from there. 

As a parent myself I would have loved to have done it. 

The communication thing [with parents] is much better, there’s more 
understanding of how it [early childhood programme] all works and what 
we do here. 

[Parent] used to drop [child] and go, now she’s really involved here, doing 
all sorts of things for us and with [child] - even gave up buying beer so 
that [child] could complete the year [when WINZ subsidies finished]. 

The teachers’ assessments of the nominated children’s progress during the year 
ranged from none or very little: 

Although he only moved into my class recently, we have seen only a little 
change in him. He is still not confident and easily distracted from tasks, 
except when his mother is here during PACTT. 

She is making slow progress, in large part due to lots of absences (52 
days). Her mother only came in about ten times for PACTT and was 
reluctant to attend these sessions. Unfortunately she [mother] only saw a 
limited range of areas like hand-writing rather than reading and maths. 



He’s a bit of a ‘free spirit’, ‘away with the fairies’ and is actually worse 
when his mum is in the room. His attendance is still very poor, mainly I 
think because his Nan keeps him at home for company. He hasn’t been 
reading at home; his mum knows it, but just doesn’t do anything about it. 
Advancements have been made, but not as many as I would have liked. 
But then with someone like [mother] you don’t change things instantly, 
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they come in with patterns well established and you don’t turn them round 
in a year, but you can see signs of some things being different. 

through ‘steady progress’: 

He’s basically a good boy who works reasonably well and is getting along 
OK. 

She’s pretty intelligent and gets a lot of support from home, so she’s 
doing pretty well, but was at the start too. 

She took a while to settle in, but is very verbal and expressive now. She 
now says ‘I love you Mum’ a lot. At the beginning we couldn’t get her to 
stay and now we can’t get her to go. 

to very positive changes: 



He’s completely changed. He was very aggressive and now has gone 
from one extreme to another. 

She [mother] is much more confident and supportive of [child]. He’s very 
excited and enthusiastic and wants to talk about the things they do 
together at home all the time. 



He’s really hooked now and his attention span is much better now. 

She’s now very noisy, very confident and has grown much more confident 
over the year. She’s now in the top group - really because of her mother 
working with her. [Mother] got a lot more confident as the year went on 
and worked with other kids as well - [child] was OK about that. In Term 
One it was very daunting having her in here, it was like having a College 
lecturer, but in the end it was nice having another adult in the classroom. 

Specific issues and suggestions for improvements that the teachers mentioned 
included: 

> organising PACTT time so it can coincide with key curriculum areas such 
as maths and reading rather than areas like handwriting and physical 
education 

> PACTT parents visiting other teachers’ classrooms 

> timing PACTT for later in the day 

> having an information sheet for teachers about how PACTT works 

> better co-ordination of topics 



MFLP summative evaluation 2004 




- 30 - 



> early childhood participation stopping because WINZ subsidies for fees 
don’t cover period outside the MFLP programme 

> longer duration for PACTT - “you just get started and it’s time to finish” 

> parents visiting, but not engaging very much with their children 

> uncertainty over kindergarten enrolments because of unknown funding 
outcomes for MFLP make planning difficult for these centres. 



2.4 Adult participants' perspectives 



All 24 of the participants at the Bairds Otara and Rowandale sites were 
interviewed at the completion of the programme. These interviews included a 
range of questions about their involvement in the programme, their evaluation of 
it and reflections on how they had changed as a result of the MFLP. The latter 
comments are reported in the following sections of the report, but general 
comments about the programme were overwhelmingly positive: 



Absolutely brilliant, I’ve never looked back, it’s just opened up so much 
and made me more aware - how to bring up my kids, how to deal with 
everyday problems, it’s taught me a whole new set of rules. It’s given me 
a path to my future. 

I’ve always been a loner, so it’s done a lot for my self-esteem ‘cos it’s 
brought me into a group of friends. I’m glad I came as an adult, you seem 
to knuckle down as an adult ‘cos you know you haven’t got much time left 
[28 year-old]. 



It’s given me another opportunity, opened a door to explore other options 
that I thought had passed me by. 

This course has opened up a whole new world for me. It has given me the 
opportunities to do the things I always wanted to do, but was hesitant to 
take the steps towards these goals. 

It’s been really hard for me, 12 I nearly walked out, but bugger it, I stayed 
for [child’s] sake. The course content has been brilliant for me - I thought 
I was doing a good job [as a parent], but I had nothing to gauge it against. 

It’s been great getting out of the house and doing something for myself, 
also making friends and meeting people like [teacher]. 



12 This site had some considerable personality clashes between some students, which 
were readily discussed in the interviews. Despite this issue, the students concerned 
persevered and acknowledged respect for each other publicly in the class graduation 
ceremony. 
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It’s been really cool, I’ve learnt lots and it’s given me somewhere to go 
and made me more out-going. 

It got me out of my isolation as a mother at home and started me thinking 
about my future - you tend to lose yourself [staying at home]. 

The parents had very few criticisms of the programme (mainly about the poor 
attendance of some students at the Rowandale site that made the class 
programme difficult at times). Suggested improvements were: 

> better Internet access for Rowandale 

> more use of AUT’s library 

> starting earlier to lessen pressure (Bairds Otara) 

> better Maori studies content in the Rowandale programme. 
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3 2004 ADULT PARTICIPANTS 



The following section reviews a range of data sources about the 2004 adult 
participants, including their attendance at the programme, academic passes, 
foundation skills of reading, writing, spelling, numeracy, speaking skills, self- 
confidence, self-efficacy, future plans and parenting. 



3.1 Attendance 



Consistently high rates of attendance in programmes in order to achieve 
duration and intensity of tuition is a self-evident, but somewhat underrated pre- 
condition for achieving outcomes and is confirmed in the research literature 
(Benseman et al., 2005). 



The graph below shows the attendance patterns for the Bairds Otara site over 
the 79 day sessions during 2004. 13 The average number of days attended for 
the total group of enrolees was 68 (85.9%) and the average number of days for 
those who completed the programme was 73 (91.8%). There was consistently 
high attendance with all of the participants, including the sole withdrawal who 
attended regularly until she shifted out of the area mid-way through the 
programme. 



Figure 7. Bairds Otara adult participants’ attendance, 2004 (n=10) 




0 20 40 60 



* withdrew in mid-June 



In comparison, the attendance patterns for Rowandale were quite different (see 
graph below). This site had much lower average attendance rates when 



13 B refers to the Bairds Otara site and R to Rowandale; the numbers are anonymous 
IDs 
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analysed for the class as a whole (79 days out of a possible 115 - 68.2%); 
however when analysed in terms of those who completed the full programme the 
average was 103 days (89.1%). In other words, when attendance of the four 
students who withdrew from the programme and one other student who had 
extended bereavement leave are disregarded, the Rowandale attendance 
patterns are similar to the Bairds Otara site. 



It is worth noting also that the Rowandale participants tended to miss blocks of 
classes (typically 4-5 days at a time), whereas the Bairds Otara participants’ 
missed days tended to be single day absences. One student at each of the sites 
achieved 100% attendance and another four had fewer than five days absent. 



Figure 8. Rowandale adult participants’ attendance (n=1 3) 




* withdrew late August 

** withdrew early June 

*** had extended bereavement leave 

**** withdrew mid-September 



3.2 Academic outcomes - adults 



As part of their programme, the Bairds Otara participants enrolled in a number of 
Manukau Institute of Technology courses throughout the year. The results of 
their course enrolments are shown in the table below (the student IDs given for 
anonymity match those given in the attendance graph). The QA level and 
number of credits for each course are given in brackets for each course. 
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Table 2. Bairds Otara adult students’ academic records, 2004 

(n=10) 



Student 


Maths 

course 


Grade 


Communications 

course 


Grade 


Prep for 

tertiary 

study 


Grade 


B1 


Intro. 

Maths 

(2/20) 


A 


Communication 

(3/22) 


A 


Study skills 
(3/3) 


A+ 


B2 


Intro. 

Maths 

(2/20) 


D 


Intro to 

Communication 

(3/22) 


A 


Lang 

Learning 

Strategies 


A 


B3 * 


Intro. 

Maths 

(2/20) 


NC 


Communication 

(3/22) 


NC 


Study 
Skills (3/3) 


NC 


B4 ** 


Maths 1 
(3/24) 


NC 


Communication 
Plus (4/18) 


NC 


Research 


NC 


B5 


Maths 1 
(3/24) 


A+ 


Communication 
Plus (4/1 8) 


A 


Research 


A 


B6 


Maths 1 
(3/24) 


E 


Communication 

(3/22) 


B 


Study 
Skills (3/3) 


B 


B7 


Maths 1 
(3/24) 


NC 


Communication 

(3/22) 


B+ 


Study 
Skills (3/3) 


A+ 


B8 


Intro. 

Maths 

(2/20) 


B+ 


Communication 

(3/22) 


A 


Study 
Skills (3/3) 


B+ 


B9 


Intro. 

Maths 

(2/20) 


A 


Communication 

(3/22) 


A 


Study 
Skills (3/3) 


A 


BIO 


Intro. 

Maths 

(2/20) 


A 


Communication 

(3/22) 


A 


Study 
Skills (3/3) 


A 



* B3 did not complete any of the Manukau Institute of Technology courses. She has been 
a student there previously and did not complete her courses then. 

** B4 was unable to enrol officially due to outstanding fees. She initially attended these 
courses informally, but then withdrew. Her tutors indicated that she would have passed 
them with outstanding grades. 



While it is difficult to make definitive statements about the grades in the absence 
of information about other students’ grades, it is noteworthy that with the 
exception of students B3 and B4, there was only one ‘not completed’ (NC), two 
fail grades (both in Maths) and that 20 of the 23 grades (87%) were in the A 
range. 

The Rowandale students were assessed for a written assignment, an 
observation study, an oral presentation and a course folder in order to pass their 
overall course. All of those who completed the course passed; because they had 
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to complete all four components, those who withdrew from the course (even in 
its latter stages) did not gain any qualification from the course. 

The Dawson Road Community site students’ academic records are shown in the 
table below. 



Table 3. Dawson Rd Community adults’ academic records, 

2004 (n=9) 





Reading and Writing 
for Academic Study 


Study Skills for 
Tertiary Education 


D1 




D- 


D2 


B 


A+ 


D3 


C+ 


C+ 


D4 


B 


B 


D5 


C 


C- 


D6 


C+ 


C- 


D7 


B- 


A+ 


D8 




C 


D9 


Started in last week of 
course 


Withdrawn due to Head 
Injuries 



3.3 Foundation skills - adults 



There has been considerable debate as to the types and validity of different 
methods of assessing foundation skills. A recent major literature review of 
literacy, numeracy and language research (Benseman et al., 2005, p. 24) 
pointed out: 



The design and use of assessment instruments are contentious not only 
in New Zealand, but also internationally. There is considerable opposition 
from practitioners and learners to the notion of assessment testing, 
predominantly because of the perceived lack of appropriate tests and a 
negative association with schooling - a spectre that most practitioners 
strive to overcome with their learners. In the US (often seen as the ‘home 
of tests and testing’), there is considerable use of LNL assessment tests, 
but this widespread practice occurs predominantly because of funding 
requirements, rather than any true acceptance of their validity or 
usefulness. Indeed, most practitioners use the tests under duress rather 
than out of professional choice. 



On the one hand, proponents of standardised tests point out that while they may 
provide specific numerical measures (i.e. quantitative data) of learner gain, they 
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are far from being definitive or unproblematic (Ehringhaus, 1991; Kruidenier, 
2002 ). 

On the other hand, the use of self-report (usually qualitative data) is often seen 
as a poor option for assessing, especially by funders. The debate among 
researchers is intense and divided as to the validity of self-report (Jones, 1997; 
Sticht, 1999). While this report is not the appropriate place to review this debate 
in detail, this brief account indicates that there is no single, unchallengeable 
approach to assessing foundation skills available at present. The approach 
taken in this evaluation therefore has been to employ a range of both 
quantitative and qualitative data sources to provide a triangulated view of what 
outcomes have been achieved in the MFLP. 

The next section therefore details the MFLP adult participants’ self assessments 
of their skills in the main components of foundation skills - reading, writing, 
spelling, maths, speaking, computer skills and library skills. The following section 
then reviews their assessments of their speaking skills and levels of confidence 
in different contexts. 



The data comes from two assessments; the first was carried out in March and 
therefore represents the participants’ skill levels at the beginning of the MFLP. 
The second assessment was carried out in November and represents their 
assessment of their skill levels at the completion of the MFLP programme. 



At the initial assessment (done in class groups), the participants were asked “On 
a one to ten scale, where one is the lowest and ten is the highest, how do you 
rate your [reading] skills right now?” This question was then repeated for each of 
the components reported below. In the post-programme assessment, the same 
questions were asked and participants were not given any information about the 
initial assessment that they had done nine months previously in order to 
minimise any influence this information might have on their second assessment. 



Figure 9 and Figure 10 show the individual participants’ self-assessments of 
their reading skills at Bairds Otara and Rowandale as well as an average for 
each site. All of the students except one (B8) 14 assessed their skills higher at the 
end of the programme than at the beginning. Four students (B3, B9, R2 and R4) 
made a gain of 2+ points on the 10 point scale during the programme; most 
moved about a point. 



14 B8 has a 6 th Form Certificate. 
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Figure 9. Bairds Otara adults’ reading self-assessments, March & 

November 




Figure 10. Rowandale adults’ reading self-assessments, March & November 




Figures 11 and 12 below show their self-assessments of their writing skills. Two 
students felt that they had made quite spectacular progress (B9 - four points 
and R1 - seven points) and a typical change was about one point. Again, B8 
was the only one who felt that she had not made any progress during the 
programme. Given that learners often have lower writing skills relative to their 
reading skills and also that it is an area in which it is difficult to achieve 
significant progress (Kelly, Soundranayagam, & Grief, 2004; Sutton, 2004), this 
perception of gain is very positive. 
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Figure 11. Bairds Otara adults’ writing self-assessments, March & 

November 




Figure 12. Rowandale adults’ writing self-assessments, March & November 




In terms of their spelling skills (Figure 13 and Figure 14 below), B3, B9, R1, R2, 
R5 and R8 all thought that they had also made progress in their spelling in 
addition to reading and writing. On the other hand, B8 felt that her skill levels 
were lower in November than in March. 
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Figure 13. Bairds Otara adults’ spelling self-assessments, March & 

November 




Figure 14. Rowandale adults’ spelling self-assessments, March & 

November 




□ March 
HNovember 



The maths self-assessments (Figure 15 and Figure 16 below) showed gains of 3 
or more levels for six students and a gain of three points on average for the 
Bairds Otara participants (including B8). With the Rowandale students, two (R2 
and R5) made significant progress, but about half made little or no progress in 
their assessments; one of those self-identified as having high skills at the start. 
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Figure 15. Bairds Otara adults’ maths self-assessments, March & November 




Figure 16. Rowandale adults’ maths self-assessments, March & November 




□ March 
ElNovember 



In terms of computing skills (Figure 17 and Figure 18 below), the Rowandale 
group felt that they had made much greater gain their Bairds Otara counterparts, 
moving about four points on average. This area is probably the one with the 
biggest gains for individual students, with B2, B4, B9, R1, R2 and R3 all making 
gains in excess of five points on the 1 0 point scale. 
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Figure 17. Bairds Otara adults’ computing skills self-assessments, March & 

November 




Figure 18. Rowandale adults’ computing skills self-assessments, March & 

November 




□ March 
ElNovember 



The last skills area was library skills (Figure 19 and Figure 20 below). Again, 
self-assessments show the Rowandale participants reporting big gains of more 
than three points on average, but especially for R1 , R2, R3 and R7. In the Bairds 
Otara group, B2, B3, B5 and B9 all made gains over three points. Four reported 
making no gain but of those, two saw themselves as having high skills at the 
beginning. 
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Figure 19. Bairds Otara adults’ library skills self-assessments, March & 

November 




Figure 20. Rowandale adults’ library skills self-assessments, March & 

November 




□ March 
SNovember 



Overall, some students such as R1, R2, R3, B2, B3 and B9 made considerable 
gains in most areas. For others, there were more uneven gains where they 
made progress in a couple of skills, but none or little in others. Such results are 
not uncommon in foundation skills, where learners’ uneven skill levels and the 
gains they make in programmes is described as ‘spiky’. 

Figure 21 below summarises the self-assessment of foundation skills by 
showing the average gains for both the sites across the six areas discussed 
previously. The graph clearly shows gains in all skill areas, with computing and 
library skills having the greatest gains. 
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Figure 21. Average self-assessments of foundation skills of MFLP adults, 

March & November 




3.4 Self-confidence 



Evaluations of general adult education programmes and adult literacy 
programmes (Benseman & Tobias, 2003) specifically have long reported 
increased levels of self-confidence as one of the most consistent outcomes for 
the learners. As one researcher on the wider benefits of learning said, “It is hard 
to think of a single field where confidence is not mentioned as a key benefit of 
learning” (Schuller, quoted in Eldred & Ward, 2004, p. 14). 



Despite the frequency of these findings, self-confidence outcomes have tended 
to be reported as somewhat removed from other ‘hard outcomes’ such as 
changes in cognitive skills and have also been consequently underrated by 
funders. More recently however, there have been more concerted efforts by 
researchers to unpack the significance of self-confidence in relation to learning 
and the broader implications of it as a learning outcome (Eldred & Ward, 2004). 



Eldred & Ward (op. cit., p. 46) concluded their study on self-confidence with 100 
adult learners and 15 practitioners by saying: 



It showed that confidence is not a static state, and that highs and lows in 
confidence can have a profound effect on learning, sense of self worth 
and activity and relationships in everyday life. Enhance confidence 
appears to increase success in learning. This in turn leads to heightened 
aspirations and progression in learning, although the nature and rate of 
this is very diverse. Further benefits include enhanced sense of self 
worth, improved ability to speak out in different situations and interactions. 
As people experienced success and began to feel better about 
themselves their sense of what was possible changed, and they wanted 
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to take on new challenges. At the same time, changes took place in 
relation to family life, friendships, social and community activity and work. 
This illustrates the deep importance of developing confidence through 
learning and suggests that more attention should be paid to this issue in 
policy, practice and further research. 

Speaking skills 



In addition to being considered a foundation skill, speaking is one of the key 
ways of demonstrating self confidence. This skill is ‘situated’ in that it varies from 
context to context. While almost all adults feel confident about talking to their 
friends, most find talking to large groups (even of people known to them) much 
more challenging. The MFLP participants were therefore asked to self-assess 
their speaking skills generally, but also in a number of different contexts - 
among friends, in small groups, large groups, to government department staff 
and to their children’s teachers. The data from these pre- and post-programme 
self-assessments done in March and November are reported in the following 
section, with a summary of the average assessments for both sites provided in 
the final graph. 



Firstly, the students were asked to assess how confident they felt about their 
speaking skills generally (Figure 22 and Figure 23 below). The Bairds Otara 
group had a higher initial level of confidence and made modest gains of about 
one point on average over the duration of the programme. The Rowandale 
participants were less confident initially, but made big gains in speaking 
confidence, especially in the cases of R1 , R2, R3 R4 and R5. Only two students 
(B7 and B8) made no progress or felt that they regressed. 



Figure 22. Bairds Otara adults’ general speaking skills self-assessments, 

March & November 
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Figure 23. Rowandale adults’ general speaking skills self-assessments, 

March & November 




□ March 
HNovember 



Not surprisingly, the self-assessments of confidence talking to friends (Figure 24 
and Figure 25 below) were the highest of all the areas assessed (six had initial 
assessments of 10) and did not show much change for many participants 
(especially for Bairds Otara). Nonetheless, even in this area, three of the 
Rowandale students (R1, R2 and R9) made big gains of over five points, while 
two Bairds Otara students (B5 and B9) moved three or more points. 



Figure 24. Bairds Otara adults’ confidence speaking to friends self- 
assessments, March & November 
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Figure 25. Rowandale adults’ confidence speaking to friends self- 
assessments, March & November 




□ March 

□ November! 



The participants’ confidence in talking to small groups (Figure 26 and Figure 27 
below) was again higher with the Bairds Otara group, although three of this 
group still made gains of three points or more. With the Rowandale group, all 
made gains, but over half made gain of four points or more. 



Figure 26. Bairds Otara adults’ confidence speaking to small groups self- 

assessments, March & November 
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Figure 27. Rowandale adults’ confidence speaking to small groups self- 
assessments, March & November 




□ March 
HNovember 



The Bairds Otara group were also more confident overall about speaking to 
large groups (Figure 28 and Figure 29 below), and about half (4) made gains of 
two or more points. The Rowandale group started with very low levels in this 
context and all made significant gains during the programme. All but one of this 
group made gains of four or more points. 



Figure 28. Bairds Otara adults’ confidence speaking to large groups self- 

assessments, March & November 
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Figure 29. Rowandale adults’ confidence speaking to large groups self- 
assessments, March & November 



10 



□ March 
HNovemberl 



R1 R2 R3 R4 R5 R6 R7 R8 Average 



The initial levels of confidence of speaking to government department staff was 
noticeably higher at both sites than for large groups (Figure 30 and Figure 31 
below). The gain at Bairds Otara was minimal for most, with the exception of B9 
and to a lesser extent, B4. The gains at Rowandale were greater, especially for 
R1 , R2 (seven points) and R8. 

Figure 30. Bairds Otara adults’ confidence speaking to government 
department staff self-assessments, March & November 
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Figure 31 . Rowandale adults’ confidence speaking to government 
department staff self-assessments, March & November 




□ March 
SNovember 



The last context the participants were asked to assess confidence in speaking 
was in relation to their children’s teachers (Figure 32 and Figure 33 below). 
Again the initial levels were high at Bairds Otara, but still showed some gain 
over the programme. At Rowandale, the initial levels were lower and made good 
gains over the year - especially R2, R4 and R6. 



Figure 32. Bairds Otara adults’ confidence speaking to child’s teacher self- 

assessments, March & November 
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Figure 33. Rowandale adults’ confidence speaking to child’s teacher self- 

assessments, March & November 




□ March 
HNovember 



Finally, Figure 34 shows the average self-assessments for both sites across the 
six contexts for speaking. The participants felt that they had made gains in all 
areas, with the greatest gains being in speaking to small groups and large 
groups. Overall, the Rowandale group made larger gains, especially for some of 
these students such as R1 , R2, R4 and R8. 



Figure 34. Average self-assessments of speaking skills in different contexts 

of MFLP adults, March & November 




Self-efficacy 



Self-efficacy is a belief in one’s ability to carry out the actions necessary to 
manage particular situations. It is more specific and contextualised to learning 
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than the more general concepts of self-confidence or self-esteem. 15 In brief, it is 
a person’s belief in their ability to learn. As the originator of the term Albert 
Bandura (1994, p. 71) puts it, 



A strong sense of efficacy enhances human accomplishment and 
personal well-being in many ways. People with high assurance in their 
capabilities approach difficult tasks as challenges to be mastered rather 
than as threats to be avoided. Such an efficacious outlook fosters intrinsic 
interest and deep engrossment in activities. They set themselves 
challenging goals and maintain strong commitment to them. They 
heighten and sustain their efforts in the face of failure. They quickly 
recover their sense of efficacy after failures or setbacks. They attribute 
failure to insufficient effort or deficient knowledge and skills which are 
acquirable. They approach threatening situations with assurance that they 
can exercise control over them. Such an efficacious outlook produces 
personal accomplishments, reduces stress and lowers vulnerability to 
depression. 



Many of the excerpts from interviews reported in this evaluation reflect changes 
in the participants’ self-efficacy. The following quotes complement them in 
relation to self-efficacy. 



I’ve changed. My confidence in writing, using paragraphs, learning new 
things, meeting new people. I used to be late, now I’m organised, 
especially with things like essays knowing when they’re due and getting 
them done. 



Look, I haven’t studied for 18 years. It’s boosted my confidence to do 
things that I already do, but I’m doing them much better. I’m glad I’ve had 
the opportunity to do this course, it’s motivated me not to sit there and 
think ‘later on’. 

The maths, just knowing that I can do it. 

It’s given me new skills. I’ve learnt some new things I never thought I 
could accomplish - communicate, maths even! 

I’ve been away from school for 25 years. It’s been a big challenge for me, 
but when I’m standing here, it’s amazing to know that the person I was 
before, I’m not now. 



The best thing has been doing things where I think I can go this high, but 
I’ve got the ability to go even higher. 



15 For further information see http://www.emory.edu/EDUCATION/mfp/efficacv.html 



MFLP summative evaluation 2004 



- 52 - 



Maths, I never liked it at school, I always wagged because of it. Now I’m 
getting good marks. I’ve been here [tertiary education] before, but I 
started too high and there was no-one there, but with this course they get 
you going, I’ve got started right. 

You do things at school, but you don’t go beyond that - here, I’ve gone 
way beyond what I thought I could do. My partner sees new things in me 
- especially my grades. He can’t believe it (he didn’t do very well at 
school either). 



3.5 Future ambitions and plans 



As the previous evaluation reports showed, the MFLP participants finished the 
year with a range of plans for the longer term. The 2004 Rowandale participants’ 
plans for 2005 were: 

> four have applied successfully for scholarships to do the Diploma in Early 
Childhood Pasifika at AUT in Manukau 

> one was applying for a sports nutrition course at AUT 

> one was moving her family to Waikato in order to do an early childhood 
course 

> one is applying for a graphic arts course 

> one is pregnant, but intends to do a fashion design course in 2006. 

The Bairds Otara group listed the following plans for 2005: 

> moving to Melbourne where she has arranged a receptionist job (“the 
course woke me up to think there are more options, so I decided I would 
do something, so this is it”), with a long-term aim of studying and then 
going into business 

> applying for a TeachNZ scholarship to do an early childhood teaching 
diploma 

> a social work diploma (“I’ve always wanted to be a social worker, but 
have had to put it on hold”) 

> enrolling in 3 year diploma in early childhood education 

> enrolling in nursing bridging course 

> enrolling in a sport and recreation diploma with the aim of becoming a 
fitness instructor or physical education teacher 

> plans to enrol in a primary teaching diploma in the second semester 

> a business management course with the aim of moving into business or 
becoming a counsellor 

> a diploma in child development with the long-term aim of becoming an 
intermediate teacher 
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3.6 Parenting skills 



As in most family literacy programmes, the teaching of parenting skills is one of 
the four key components of the MFLP. 16 These skills have been taught in a 
variety of ways in the different sites to date. At both sites, they have been an 
integral part of the child and human development curricula, PACTT, as well as 
an important part of the informal discussion and debate that emerges in the 
course or running programmes for 30 hours a week. 



Measuring the impact of the MFLP on parenting skills is not straightforward. 
Changes in parenting behaviours (especially ones that are thoroughly 
embedded in previous generations of families) do not occur readily or even in 
the short term. For this report a number of data sources have been used to 
document changes in this area, including key informant interviews (especially 
teachers and principals) and the parents themselves. 



3.7 Parents' perspectives 



Interviews 

As part of their end-of-year interviews, the participating parents were asked what 
they felt the programme had achieved for themselves as parents and their 
families generally. Their comments in response to this question included: 

[When asked what aspects of their family life had changed as a result of 
MFLP, this student first looked very pensive and then started crying 
before giving this answer] It’s just changed my whole life. I used to smack 
and belt them (it’s not something that I’m proud of), now I talk to them. I 
send them to their rooms to cool down, then talk to them and try to work 
things out with them, how to change things. My kids tell me all the time 
that I’m different now. 

I’m doing an anger management course ‘cos I don’t want to yell at my 
kids any more [this was prompted by a listening skills module]. 

We did a project on positive disciplining - I used to smack something 
awful. But now I’ve tried ideas on my son, using a chart to record 
behaviour, rewards and that. Now he listens, it’s worked really well for 
both of us. 



16 It is the parenting component that generated the most criticism as ‘deficit’ models of 
provision - see Benseman (2002, p. 5). 
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I have a lot more patience with my kids now. It’s taught me to stop and 
listen, I was always too busy before and I find them easier to manage 
now. 



I was a bit of a slacker to be honest. Things I used to do with [name] I 
don’t do with [nominated child]. I used to be a discipline person, now I do 
better things. 



I’m more patient with them, I respect them more and I listen to them. It’s 
calmed me down as a parent. I used to yell a lot ... they seem to 
understand me now, we still argue, but we laugh a lot too. 

It’s made me focus more on education and given me insight into her 
learning, especially through watching teachers in PACTT. 

It’s taught me how to look after my kids properly, how to communicate - 
especially when they are attention-seeking. I never used to see it, but this 
course showed me all about it. 

It’s made aware of how I was bringing up my kids. I’ve learnt to discipline 
myself and not them. It was me at fault, not them, I’ve got so much more 
patience. We’re much more involved in school functions now, we didn’t 
bother before. We do family weekends that we didn’t do before. We plan 
things, go to the beach, the zoo. 

One parent was frank that she did not think that the programme had helped her 
nominated child, but that her relationship with her older children had improved. 



To be honest, I don’t think it’s made much difference with [nominated 
child]. At home I try my best, but my kids treat me more like a brother or 
sister than a mum. But it is different with the older ones. Now I sit down 
and talk to them, I no longer scream at them. The 1 4 year-old sees me as 
a different person, she’s not screaming at me, so we talk. She says that 
she saw me as someone who just sat at home, but now, ‘Mum, you seem 
to be listening to me.’ 



Many of the Bairds Otara parents specifically mentioned the value of doing the 
Child Development course. They found the course particularly relevant to their 
parenting roles and issues by giving them a good understanding of how and why 
their children develop in the ways they do. 



It’s shown me why they are the way they are. [Talks in some detail about 
being over-protective about her first-born child, now 14 years old]. It’s 
made me realise I’m not the only one who thinks and does things like that, 
there are others like it too. 
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One particular aspect of their parenting role that a number of the participants 
mentioned was their new-found ability to help their children with their homework. 

The kids used to come home with their homework and I couldn’t help 
them. You want to help, but you can’t - I used to feel so put down. Now I 
have ideas about how to help them ‘cos I understand it myself, especially 
stuff like maths, 



It’s so neat being able to work with them, showing them how to do things 
and that. I used to keep out of it before because I didn’t know myself. 



I never used to be able to do it, stuff like algebra, ‘cos I’d never done it 
before myself. 



It’s great being able to watch the teacher [during PACTT] and then try 
those things at home. 



I’ve got to be with my children when the need me like homework. Before, 
I’ve been with them, but not like now. Things are clearer for me. I could 
only help them with the easy things, now I can understand what they’re 
doing, that’s the beauty of it. 

Being able to understand their homework, it just opens up a lot of cells in 
your brain. 



You need to relate to your child, to understand what they’re doing - 
especially in things like maths, fractions and that. 



I didn’t want them [younger children] to follow in the same footsteps [as 
older children]. They used to come in, have some food and watch TV and 
not get round to doing their homework. I needed to show them how 
important it is to be at school by being there - we can both do the same 
things now. She’s improving a lot, especially the way she communicates 
with the teacher. 



While for others, it has become important to model educational practices for their 
children. 



Like this [interview with evaluator], I could never talk to anyone. 
Everything is ‘more’ now. Now I’ve got kids, I have to get an education, 
before it didn’t matter. 
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Learning journals 

The Bairds Otara participants kept learning journals as part of their Child 
Development course. These students agreed for the evaluator to have access to 
these journals and used them anonymously in this report. They are a useful 
source of data on how the learners’ perspectives change over time and also how 
this course influenced their parenting beliefs and behaviours during the year. A 
selection of journal excerpts is given below. 

Some journal entries illustrate how the parents saw what influenced their child’s 
ability to learn at school. 

[Child] was doing well in class today. She was doing maths by herself. 
She could do things by following the instructions she had from the 
teacher. In this way I can see how she improves from day to day by 
listening and following the instructions she had been given. She could 
also help her classmates in working as a team. This helps her understand 
things in class. 

Other excerpts recount their on-going struggles to change parenting skills. 



[Child] moves back to the [train] track, he tries to grab a toy off [other 
child] then hits her for not giving it. I get all frustrated because I’m trying to 
teach him not to hit, to try and use his mouth. So in a situation like this I 
just walk away because he’s not listening to me. 



Or resolve personal issues (with support from fellow classmates). 



[Child] is all clingy today and it took me a long time to get away from him. 
I’m finding it hard, I don’t know why he is behaving like this. I came to it as 
a result of discussing it with the other friends [on the course]. [Child] is 
lonely being the only child. Him and [friend] have this brother/sister 
relationship. He used to get his way with [friend], but ever since she’s 
been saying ‘NO’ to [child], he doesn’t have that dominant power over 
her, so he’s been trying it on me. And when you push the wrong buttons 
with [child] someone ends up crying and it’s not me. [Fellow student] 
suggested I made the first step coming on this course and now I have to 
make the first step for him. A brother or sister for him, that is the question. 
I have just figured out that what I wanted to do and if I decided to have 
another child, what I wanted to do would have to be put on hold, plus 
losing a child last time was hard enough. I feel sad for [child] being by 
himself, I can understand why some of the things he does are driving me 
mad. I cried because reality has hit me in the face with this situation. 
What do I do? Do I have another child and put my future on hold or do I 
deal with this up-coming situation, knowing that he’s lonely? 
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The following series of entries shows how one parent had learnt to apply 
concepts from her course to help her son and also the ups and downs of a 
child’s progress - but also the joint efforts of the teacher and the parent trying to 
overcome the downs. 



April 1 7: today the whole middle syndicate classes had split up into 
groups and went into different classrooms for maths time. [Child] went 
into Room X, the teacher was playing a game with a deck of playing 
cards. The children had to learn to identify numbers and counting up to 
15. They also had to learn to identify numbers from the lowest to the 
highest. The teacher unscrambled four cards for [child], he had to put the 
cards in order, which he did correctly and fast. He showed no hesitation, 
he was eager to do the cards again. He showed a big smile for doing it 
right. He also showed three basic stages in learning: taking in information, 
storing information and recalling the information. I just learnt it today in 
Child Development about cognitive development. 



April 24: today in maths [child] showed the three stages of memory. On 
the weekend I had shown my children number counting methods and 
today [nominated child] showed all the stages of memory. He did his work 
fast and correctly because of what he had learnt and remembered. He’s 
more happy when he knows what he’s doing and especially in maths. 
When he doesn’t know, he shows trial and error, but teaching him on the 
weekend had helped him with his maths. 

May 27\ today [child] had a test with his teacher on his maths levels. The 
teacher reviewed him on all the methods he has been learning. He had to 
do all the stages of memory, he had to shuffle in his schema on what he 
knows or on what he has learnt. He passed the test, the teacher said he 
can move up to the next stage into another group. I’m proud to see my 
son develop a good learning strategy in his maths. 

June 6: the lunch lady made a noise down the back of the hall by opening 
up the door to the lunch-room. This made all the children turn to see what 
was that noise? This made me think about what we had learnt in class on 
primitive reflexes and it just happened today at Bible Study. The children 
and the teachers, also ourselves, reflexed to stimulus, an involuntary 
response to a sound. This was a laugh for [fellow student] and I, we both 
looked at each other and said, ‘REFLEX’. So we acted this scene for our 
role play in class. 



June 30: this week and last week [child] has developed great learning 
methods throughout his maths time. He has developed a lot in his 
cognitive development for a seven year-old child. It is really good to watch 
my child increase his fine motor skills and gross motor skills through his 
growth. When I first watched him in the beginning he showed confusion 
and a lack of understanding, but through this term in helping him with his 
maths and reading and maturing he has shown a lot of the stages of 
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sequences of cognitive development. It has been a great progress and 
this has given him confidence and understanding in everything he does 
now. Since he shows confidence in himself he now says, ‘I know Mum’, 
which makes me laugh and happy to see him have that confidence in his 
maths and play and himself. 



[following a new teacher, the child has lost some confidence in his maths] 



November 18: maths time and spelling with [teacher] today. [Child] played 
up today because he couldn’t get the answers for the maths test. I tried to 
help him remember the methods he had learnt but he couldn’t and started 
getting mad and frustrated with himself and started crying. I started giving 
him positive talk telling him ‘it’s OK to get things wrong’ and ‘it doesn’t 
matter you can try again’ and ‘it takes a lot of practice” and ‘you’re going 
to get it right next time’. [Teacher] gave him the same talk, trying to 
encourage him. It made him a bit better, until he played the maths game 
and got over it very quickly. Typical boy, when it comes to games, they 
stop sulking! 

The next series of entries demonstrates a growing confidence in a parent’s 
confidence in her daughter and the enjoyment of working alongside her. 

June 6: I am happy with the way my daughter is doing in school. I can see 
her improvement, she is making with having my support around. I hope 
that she will continue to be like this as she grows older. 

June 10\ getting to know the changes in working along with my daughter 
encourages me to keep on going with my study. Helps me on my weakest 
point too, specially with the different tasks we are doing. 

June 1 7: sitting back to help my daughter everyday in PACTT draws me 
closer to my kids. Also learning at the same time as my daughter. 

June 21 : a well performed day for my daughter. She has done her 
exercises well before time. I was amazed with her. I gave her a hug and 
patted her on the back - she was overwhelmed. 

June 22: teaching kids the right path to go at home and learning off them 
is a bonus to me. Hoping that I would continue doing it for her in the 
future is what I am dreaming. 



June 26: see your own kids improving with their study uplifts your 
loneliness and confusion. Happy to work with my daughter in her class 
with her maths every day. Shows me she has the ability to succeed in 
whatever career she will pick when she is old enough. 
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August 4: after our long two week holiday which was good to have a 
break from study. I enjoyed being together with all my kids. Now I have 
started study again, I have to be with my daughter in class for half an hour 
a day. So this morning she was doing her maths. It was good to work with 
her doing maths, also because I was doing the same, which was 
fractions, which really interest me to learn and teach her at the same time. 
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4 2004 MFLP SCHOOL-AGED NOMINATED CHILDREN 



This next section examines data about the nominated school-aged children who 
participated in the MFLP during 2004. Most of the data presented is of a pre- 
and post-test comparison, but is supplemented by some qualitative data from 
their parents and their teachers. 17 



In order to give some degree of comparison, data from a ‘control’ child is given 
wherever possible. This methodology was used in a British study of family 
literacy (Brooks et al., 1997) and involves the identification of a second child in 
the same class (but not participating in the MFLP) who best matches the MFLP 
nominated child in terms of their academic levels at the beginning of the year, 
gender, ethnicity and social background. 18 Although this form of control 
comparison is somewhat crude, Brooks et al. argue that is a reasonably valid 
method of isolating effects on the children of the family literacy programme, 
therefore provides a useful comparison. 19 



17 The only available data on pre-school age nominated children (the observations and 
perspectives of parents and teachers) was presented in the previous section. 

18 Achieving matches across the last two factors is not difficult in most cases because of 
a high degree of homogeneity in the participating schools. 

19 Whether MFLP has a small or large impact on children’s learning compared with other 
school literacy activities is unknown. 
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4.1 School attendance 



The first factor to compare the MFLP participants with their control counterparts 
is attendance (Figure 36 below). 20 The graph shows very few differences 
between the two groups in their attendance patterns, with the average for MFLP 
children of 88.7% and 89% for the controls. 



Figure 35. Rowandale MFLP children and controls’ attendance totals, 2004 
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Figure 36. Bairds Otara MFLP children and controls’ attendance totals, 2004 



1 oo% 
80 % 
60 % 
40 % 
20% 
0% 



□ MFLP 
^Controls 




r <? 5? & e > 



20 In these figure IDs R refers to Rowandale, 6 to Bairds Otara and Cto controls. The ID 
numbers refer to the same children throughout the report - i.e. R MFLP 6 is the same 
child each time. 
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B MFLP 3, 6 & 7’s parents had either poor attendance at MFLP or were absent for sustained 
periods during the year 

4.2 Reading 



Developed by Marie Clay, Running Records are commonly used in New Zealand 
schools as a “framework for systematically observing a child’s reading 
behaviour” (Ministry of Education, 2003b). They are seen as an appropriate 
method for monitoring early reading development in a child’s first few years at 
school and the results provide a reading instruction level. 21 The observations 
cover oral language, concepts about print, reading of continuous text, letter 
knowledge, reading vocabulary, writing vocabulary and hearing and recording 
sounds in words. There is a general expectation that a new entrant will progress 
approximately 14 levels after a year of classroom instruction. 22 Level 15 equates 
to a 6.5 reading age and as students progress up the levels, progress between 
the levels takes longer. 



Figure 37 below shows the Running Record levels for both the Rowandale 
MFLP nominated children and their controls recorded at the beginning and end 
of 2004. Most of the MFLP children clearly made better gains than their controls, 
especially R MFLP 3 and 4 who made 10 and 11 levels gain respectively. The 
only exception is R MFLP 6 who made no gain at all over the year. The average 
gain for Rowandale MFLP children was 5.3 levels and for the controls, 2.5 
levels. 



Figure 37. Rowandale MFLP children and controls’ Running Record Levels, 

beginning and end of year 




* This control was the closest match available. 



21 Levels are taken from the PM Benchmarking kits 

22 Private communication, MoE literacy leader, March 2005 
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Four of the Bairds MFLP nominated children gained 10 or more levels. The 
average gain for Bairds MFLP children and the controls was 8.4 levels. B MFLP 
3, 6 & 7’s parents had either poor attendance or were absent for sustained 
periods during the year; despite this, two of these children made similar or better 
progress than the controls. 

Figure 38. Bairds Otara MFLP children and controls’ Running Record 
Levels, beginning and end of year 




Running Record Levels 



4.3 Maths 

Progress in maths was assessed using Maths Strategy levels. Figure 39 below 
shows the before and after assessments for the Rowandale nominated children. 
Here, there was less movement between the levels over the year (albeit on a 
smaller scale of levels than with the Running Record levels), but the average 
gain for MFLP children was 1 .3 levels and for controls, 1.1 levels. 
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Figure 39. Rowandale MFLP children and controls’ Maths Strategy Levels, 

beginning and end of year 




Maths Strategy Level 



Figure 40 shows the comparable data for the Bairds Otara children and their 
controls. Again, there are not big differences between the MFLP children and 
their non-MFLP controls. The average change of 1.1 levels overall is the same 
for both groups. 



Figure 40. Bairds Otara MFLP children and controls’ Maths Strategy Levels, 

beginning and end of year 
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B MFLP 3, 6 & 7’s parents had either poor attendance at MFLP or were absent for sustained 
periods during the year 
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4.4 Parents' assessments of children's progress 



As part of their end-of-year interviews, the parents were asked what they 
thought the programme had achieved for their nominated children. Consistent 
with teachers’ and principals’ perspectives, there was a range of views about the 
amount of progress that parents felt their children had made over the year. 



Some felt that their child’s progress had been satisfactory prior to coming into 
the MFLP and that this pattern had continued during the programme, albeit with 
some additional benefits. 



He’s always done reasonably OK and had lots of things to help him, 
although I do think it has helped him socially. 

She’s always been at the top of her class, so she has real pride having 
her mum in the class showing me what she’s done. 

Others felt that their children had made considerable gains, especially in terms 
of their attitudes towards learning. 

He had been finding his homework hard, but now he’s doing it much 
easier and he’s less grumpy with it. 

She’s changed - she always looks forward to me working with her. She’s 
improved a lot, got motivation to do her homework straightaway after 
school without being asked. 

He’s learnt different ways of communicating. He used to be physical, 
hitting his sister, in trouble quite a lot. Now he’s learnt to use his mouth, 
not his fist. 

He’s changed a lot. He used to be an observer, now he watches and then 
joins in. He speaks a lot, didn’t used to at all and he now says ‘I’ve got 
friends’. 



She reads heaps, makes me go to the library. She’s more advanced than 
my older one, ‘cos I never read to her. I thought reading was nothing eh? 



Everything has come up so much from where he was, especially to do 
with me working with him. He’s much more involved in extra-curricular 
stuff, getting into school activities now. 



Her writing book wasn’t at all clear at the beginning, but is great now. 
She’s talking clearly and lost all her shyness. 
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She talks to her classmates, she was very shy to start with and me being 
in class helps. 

And making transitions, 

It’s the first time he’s been to kindy and he’s been able to settle in really 
well with me being involved. 



And a few felt that they had not changed much at all in the short term, although 
there were indications of improved social skills. 



He still plays up a lot. When I walk in, he thinks he can do what he likes. 
PACTT hasn’t worked that well for him. But it’s still great being in class, 
seeing how teachers talk and relate to children. I want to take what they 
do and do the same. 

He’s coming out of his shell a bit, being round him in class. I can see the 
things I need to work on now by watching him in class. 

[Name] is still doing the same things, but she can play with the other kids 
now, she used to just sit back and watch. 



4.5 Early childhood educators' perspectives on child participants 



The early childhood teachers involved in the project were also consistently 
positive overall about the programme for the nominated children in the 
kindergartens - “there’s definitely been a difference since it started.” The main 
changes they had noticed in the children were their growth in confidence, 
greater independence and improved oral skills. 

One group of kindergarten teachers said that most of the parents were very shy 
initially, but had gradually become more comfortable in visiting during the day 
with PACTT. In particular, they felt that sustained periods in the kindergartens 
were especially valuable in exposing the parents to the modelling of teaching 
methods by the kindergarten teachers. They listed a number of changes they felt 
that had occurred as a result of the MFLP: 

> parents were now staying on outside MFLP to help with programme 
activities 

> parents were staying longer to work with their children 

> they were asking for their children’s artwork to take home 

> they were also making comments about their artwork and its development 

> they were noticeably more interested in what their children were doing 

> mutual support among the parents over issues that had arisen. 
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In addition to the changes for the children and the parents themselves (their 
motivation, life skills and job skills), they felt that the family literacy parents’ 
involvement was beneficial for all of the kindergarten children because of their 
active involvement in the programme - “it’s had a ripple effect among whanau 
and the community, other parents see the [MFLP] parents doing the course and 
hear us talking [about MFLP], so they ask questions about it.” 
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5 2003 MFLP PARTICIPANTS 

Many changes that occur as a result of education do not always happen in the 
short-term. Sometimes learners do not immediately appreciate the relevance of 
their new-found knowledge and skills, sometimes opportunities to apply the 
knowledge and skills do not eventuate and more commonly, other events and 
issues prevent learners from acting on their new-found goals and ambitions 
(Benseman & Tobias, 2003). Learners may need to put their new plans on hold 
in deference to others’ needs and issues (such as health crises or financial 
constraints for example). Despite the self-evident nature of this observation, very 
few summative evaluations look at impact on participants beyond a very short 
period after the completion of the programme. In most cases, this restriction is 
due to the pressure to show results to funders and also that the emphasis is 
usually on changes in specific knowledge and skills rather than the broader 
impacts that can occur, particularly in programmes like family literacy. 

In recognition of this dimension, this evaluation has also included a follow-up of 
the participants from the 2003, who had been out of the programme for at least 
12 months and in some cases, 18 months. 



5.1 2003 adult participants 



There was a total of 37 adult participants in three intakes (two at Bairds Otara 
and one at Rowandale), seven of whom withdrew at some point during the 
programme. Of the 30 who finished, we were able to interview 16 participants. 
Most of the interviews were done by phone because of difficulties in locating the 
respondents and finding suitable times for interviews; because of their previous 
contact with the evaluator, the lack of face-to-face contact was not seen as a 
difficulty for the interviews. 



Because of the diversity of their experiences the data from the interviews is 
presented as a series of individual vignettes covering the most pertinent points 
from the interviews to illustrate what these former participants have done as well 
as their reflections on the impact of the MFLP since completing it. 

A 23 initially enrolled at a tertiary education course, but had to withdraw 
because of family issues. She then worked as a teacher aide and as a 
community support worker with special needs children for the rest of 
2003. As a result of working in this area, A has clarified that she wants to 
work with primary-age children and is planning to enrol in a Diploma in 
Teaching course in 2005. A felt that the MFLP had been “a great stepping 
stone for me, it opened my eyes.” She says that “I carry myself differently 
now, I project myself, I speak more eloquently and words are more 



23 To preserve anonymity some details have been generalised and the participants are 
identified by letters of the alphabet; the four male participants are referred to as she. 
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important for me now.” Talking about the MFLP, A said that it contributed 
to a very important turning-point in my life. It taught me that if you want 
changes, you’ve got to get involved.” A feels that her child is doing really 
well at school - “she is a lot more communicative, ‘cos she would have 
been hiding under the table otherwise.” 



B is now eight months pregnant and has put her plans to work for a social 
service agency as a secretary on hold for now. Reflecting back on the 
MFLP, she thinks that it was “just awesome. It changed my attitudes 
towards my kids, I’m just so much more patient with them, especially 
working with them on their homework. I’m not yelling at them like I used 
to.” B thinks that her daughter is finding her schoolwork much easier and 
she enjoys working with her on her homework. 

C has spent the past year playing a strong support role for her whanau, 
including caring for two relatives with terminal illnesses. She has also 
been enrolled in a free te reo course offered by an out-of-town 
polytechnic, which she says has been important in understanding “where I 
come from and who I am.” C misses the MFLP - “it brought my inner me 
out.” While she says that she still has problems with maths (“I just can’t 
seem to retain it”), she says that her reading, spelling and writing are now 
of a level that she can manage on her own and is no longer dependent on 
her husband for help. C’s grand-daughter was her nominated child in the 
MFLP; I feel that she stills “gets a bit distracted” but her reports are 
excellent. She feels that taking an interest in her grand-daughter “was 
really important at that time.” She regrets that she had not been able to 
do the course herself when she was a young mother - “if I had known 
what I know now when I was a mum, I wouldn’t be where I am now.” C’s 
plans are to continue in her support role for her whanau (“they really need 
someone you know”) - and the te reo course. 



D has been doing relief teaching for an early childhood agency over the 
past year, which has helped confirm her interest in working with this age- 
group of children. While she had originally planned to start her ECE 
diploma in 2004, she had to defer it as she had trouble finding suitable 
childcare for her pre-schooler, but she is hopeful of enrolling in 2005. Her 
reflections on the MFLP were mainly about its value of showing her the 
value of working with her children and giving her to confidence and skills 
to do this. D also felt that the programme had prompted her to review how 
she disciplined her children - “I always used to smack them, but now it’s 
good ‘cos I know what to do that’s better.” 

E did what she said that she intended to do at the completion of the 
programme - she has a full-time job working as a packer in a factory. 
Although she has been quite sick recently (requiring a hospital stay), both 
she and her husband are both now in jobs for the first time. She feels that 
her daughter is very happy at school now and “has done really well.” E 
also reported that she had changed how she interacted with her children, 
sitting down to talk to them much more now. 
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F has just completed her first year of a social work degree at a tertiary 
institution. The year’s study has not been without its problems and 
challenges however. She passed seven papers, but failed one due to 
health problems (an on-going medical condition). At one point she said 
that she nearly withdrew due to her health, but also “not knowing how the 
system works” - especially considerable enrolment difficulties and paying 
fees. During this period she got a lot of support not only from the 
institution’s Pasifika Centre staff, but also MFLP staff who have 
maintained contact with her since she left the programme and supported 
her through her selection interview. F is effusive about how the MFLP 
changed her perspective on herself as a learner - “Being out of school a 
long time, I never thought that I could get back into education, but family 
literacy changed all that for me. I didn’t believe in myself until I went there, 
about what I could do.” F’s grandchild was her nominated child in MFLP, 
who unfortunately had to return to their Pacific homeland. The child is not 
enjoying schoolwork there because of the different teaching approach 
taken. F is clear that her tertiary study has not been easy, but is confident 
her second year will be easier because of her better understanding of 
“how it all works.” As she says, “I’ve started on a journey, there’s still lots 
of issues, but I’ve started on my trip.” 



G has completed one year of a business diploma at a polytechnic. She 
has passed everything to date and is contemplating starting on an 
accounting B Com in 2006. G was one of two MFLP students who were 
awarded Outstanding Adult Learner of the Year awards for the national 
Adult Learners’ Week in September. G had tried tertiary education before, 
but had been unsuccessful. She feels that MFLP had made the difference 
for her this time round because “they give you more detailed help, so it’s a 
good step up - I went into it much more confidently this time.” Her 
nominated child has made a successful transition to school from his 
kindergarten and is doing well in his new environment. Talking about what 
MFLP has meant for her, G said that “as a parent I used to like them to go 
to school, but no more than that. Family literacy helps parents participate 
in school. The teachers are calling for parents to come in and the children 
see it’s important. They like to see us [parents] come in and be around 
the school. My [14 year-old] son says that he sees that I can do it and 
now I’m going to do it too. [He says] My mum’s famous [for the Adult 
Learner Award] ‘cos I can see her doing all this. They’re always asking 
me, ‘Mum are you proud of me?’ when they get awards at school.” G has 
not found her new direction easy, as reflected in her story of a discussion 
about stress in one of her courses. When she listed all the stresses in her 
life (church, part-time job, family, tertiary study), a fellow student 
commented on how much stress she had in her life - to which G replied, 
“yes, but it’s positive stress, ‘cos I’m doing very good things for my 
children, in my heart I want my children to follow my footsteps, even 
though it’s not easy.” 



H has been working as both a cleaner and a teacher aide at her MFLP 
site school. She finds working with special needs children very rewarding 
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and “just loving it. You never know what’s going to happen, they’re just 
great.” She plans to continue this work in 2005 to gain additional hands- 
on experience and then embark on a special needs qualification. H says 
that MFLP boosted her confidence, helped her make friends and gave her 
insight into cultural differences. H is clear that the teacher aide work has 
been the appropriate step for her at this stage - “If I had jumped straight 
into study, it wouldn’t have worked, so this is a good medium step. The 
experience here is just awesome, especially with [husband] back in work 
now, managing the kids and everything else. H’s daughter is doing well at 
school (she is now reading at her age level for the first time), although she 
is finding it difficult without her mother coming in for PACTT. H had 
chosen her as her nominated child “because I didn’t know how to teach 
her. I learnt a lot just from watching her teacher - I had been doing things 
wrong for her.” Overall, H found the programme to be of tremendous 
benefit - “I was a stay-at-home mum, doing a bit of housework, you know. 
My husband got a job two months after I started family literacy [his first in 
nine years], so it’s all go now.” 

I has also been working as a teacher aide at the same school as O as 
well as working for a social service agency. She sees this move as “taking 
time-out really” at this stage while she re-assesses her future direction. 
She had originally intended to do a teacher training course, but is now 
contemplating a bachelor’s degree in child psychology (“it’s always 
fascinated me, but I never knew that I could do it or not until I went 
through family literacy”) and some distance education courses in 2005. 
She has a long-term goal of working with youth as a counsellor. 
Previously at home on the Domestic Purposes Benefit, I was glad to have 
done the MFLP - “it was all about getting out and doing something about 
it [being at home]. She particularly valued learning how to communicate 
better with her teenage sons. Her younger son, who was in the 
programme, has continued to make good progress at school, especially in 
reading and writing. I stood successfully for the school Board of Trustees 
election in 2004 - something she says she would never have thought of 
doing previously. 



J worked part-time in two jobs in 2004 - one with a pre-school and the 
other at a mail centre. She had planned to enrol in an early childhood 
diploma in July 2005, but her husband has been sick and she is also 
expecting a child early in the year, so has put these plans on hold for 
now. J says that she is excited about being pregnant again - “it’s just 
excellent that I’m having another chance to do it all different.” Talking 
about the MFLP, she says that “it gave me a lot more patience, it opened 
my eyes to all of that child development. When you’re working, you don’t 
have time to do that, but now I see it how a child sees it.” She related how 
a relative had commented to her recently, “you’ve got a lot of patience 
with them to do it like that [giving answers the children could answer].” J 
was particularly pleased that her relative had noticed the change. She 
says that her nominated child is making great progress; she is especially 
pleased that he is more focussed in his schoolwork and gets his 
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homework done straight after school - something that did not occur 
previously. 

K is also pregnant, with the baby due in early 2005. She says that she is 
spending more time with her children and especially with their homework. 
She is not clear about what her long-term plans are at present. 



L has just successfully completed her first year of a diploma in social 
services at a polytechnic. She says that she coped reasonably well due to 
the preparation that MFLP provided, although she did find the te reo 
component difficult. She sees the MFLP as “one of the most important 
things in my life, it gave me a focus on the future. I had tried to learn at 
home, but it didn’t make sense, I didn’t know what to do.” She had been 
“doing nothing much at home - I had wanted to do a course, but didn’t 
know how to do it.” Prior to her involvement in family literacy, L said that “I 
didn’t want to make my children first, I wanted [partner] to control them. 
It’s working now.” L feels that her daughter is doing “OK” compared with 
before the MFLP. 



M had recently given birth to another child just prior to the evaluation 
interview. She was very complimentary about the value of MFLP for 
herself and her family. She feels that her son has benefited from it; since 
starting primary school in May, he had received three awards of which 
they were immensely proud. 



N was also awarded an Outstanding Adult Learner of the Year Award and 
stood successfully for her school’s Board of Trustees election. In 2004 
she successfully completed the first year of a B Ed qualification to 
become a primary school teacher. She has found the level of her study 
quite manageable, but managing the workload of completing assignments 
etc. on top of her family obligations as a single parent very demanding. N 
admits that she nearly gave up twice - “but then when I think about why I 
wanted to get into this, I want to get a better life and that’s what I’m about 
now. That’s what kept me going. I get $1 3k on the benefit, but I’ll get $42k 
as a teacher - why wouldn’t I want that?” N says that she learnt to cope 
with the demands of her study by getting strong support from her brother 
and parents who have provided back-up childcare when needed - “we 
went into this together, I said ‘I need your support for three years if I’m 
going to get through this.” She now feels confident about being able to 
complete her bachelor degree and is starting to think about a masters or 
even a PhD in the future. N says that she had been “at home on my own, 
not wanting to work or anything really” before she enrolled in the MFLP 
and that the programme had prepared her “for everything that I’ve done 
so far.” She feels that her daughter is doing “not too bad” at school, 
especially in terms of wanting to read more (“she never wanted to 
before”), although it has been hard for her at times because of the 
demands on T. 
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0’S daughter was in hospital for an extended period in 2004, which 
curtailed her activities somewhat. She had found it difficult “to get back 
into things” at first, but had done a free computer course “to keep things 
going.” She had recently heard that she had been accepted for training as 
a corrections officer, starting in February, 2005. She is very excited about 
this success, as she said “I’m not very good at study and I think this suits 
me better.” The value of MFLP for O is that “I know what the kids are 
going through with their homework. I take much more notice now about 
their homework now that I can help them. They used to come to me and I 
wouldn’t know what to do ‘cos everything is so different. Now they come 
home and we get it done straight away.” O is very happy with her 
daughter’s progress. 

P has been working full-time as an early childhood reliever and has 
enrolled for an early childhood diploma starting in 2005. She says of 
family literacy, “it was just a great start for me, it’s given me the 
confidence to carry on.” P feels that her son has continued to make good 
progress at school, which he was doing before enrolling in the MFLP. P 
stood unsuccessfully for the school Board of Trustees election. 

Of those who could not be located: 

> six of these had withdrawn from the programme during 2003; of these, 
one had gone overseas, three had moved out of Auckland and two had 
changed their addresses and phone numbers 

> five had changed addresses and/or phone numbers 

> four could not be contacted despite a minimum of four phone calls. 

Flowever some information about seven of the above group was picked up in the 

course of interviews with their fellow students: 

> Qhas shifted to another part of Manukau and now has a full-time job 

> R enrolled in a Manukau Institute of Technology course, but withdrew 
and has now left the area 

> S has been nursing her sick mother in her home town 

> T enrolled in an Manukau Institute of Technology course in the second 
term of 2003 

> U has moved out of Otara and is working night shifts 

> V is currently working part-time and is considering applying for an early 
childhood diploma course 

> W has moved out of the area and working as a cleaner. 

Other points that were mentioned by a number of the interviewees included: 

> The tremendous importance of the MFLP adult educator who has stayed 
in contact with these former students, offering them support, advice and 
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always conveying an absolute conviction that they are capable of 
success in their tertiary studies 

> The value of seeing alternative ways of managing their children’s 
behaviour (especially in terms of disciplining), even though it is 
fundamentally different from how they were raised - “just because that’s 
how we were brought up doesn’t make it the right way.” 



5.2 2003 MFLP nominated children 



In the case of the nominated children who participated in the 2003 programme, 
the teachers who taught these children in 2004 were also able to provide their 
Running Record levels at the beginning of 2004 and at the end of that year. As 
with the children of 2004 MFLP participants, the teachers also nominated 
another child in the same class who best matched the 2003 MFLP nominated 
child in terms of their gender, ethnicity, social background and initial assessment 
levels. The comparisons for both groups of children for their Running Record 
levels, Maths Strategy levels and attendance records are given below. 



The Running Record levels in Figure 41 below show little difference between the 
MFLP children and their control counterparts except for R MFLP 6 and 7 where 
progress had been greater for MFLP children (particularly R7). The average gain 
in levels was 7.5 for the MFLP and 5.6 levels for the controls. 



Figure 41.2003 MFLP Rowandale nominated children’s Running Record 

Levels 




Running Record Levels 



Figure 42 below shows the comparison in terms of their Maths Strategy levels. 
Overall, there was not a lot of change in the levels (apart from MFLP 4 and his 
control), with three children making no change in levels at all. The average gain 
for MFLP children was one level, while the controls gained 1.5 levels on 
average. 
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Figure 42.2003 Rowandale nominated children’s Maths Strategy Levels 




Maths Strategy Levels 



Finally, in relation to attendance during 2004 (Figure 42), there are very few 
differences, with the average figures of 90.6% attendance for the MFLP children 
and 90.5% for the control counterparts. 



Figure 43. Rowandale 2003 MFLP children and controls’ attendance (%) in 

2004 
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BROADER ISSUES OF MFLP DEVELOPMENT 



This part of the report discusses the stakeholders’ perceptions and evaluations 
of the role of COMET as the lead agency for MFLP, the issues and challenges 
that have faced the programme since its inception and how the model might 
develop in the future. 

Seventeen key informants in different roles were interviewed to inform this 
discussion, including lead teachers, principals and members of Boards of 
Trustees in schools, managers and liaison staff with the tertiary providers and 
early childhood centres, adult educators and COMET staff. Sampling continued 
until there was data saturation (i.e. no new themes or perspectives were being 
added). 

6 COMET'S ROLES DURING THE PILOT 

Respondents described COMET as having played a number of very important 
roles during the pilot that were different from the work carried out by other 
partners. 

6.1 Advocacy and leadership 

All respondents talked about the importance of the roles COMET had in 
instigating the MFLP programme and bringing the partners together, particularly 
at the pilot stage. Programme partners clearly saw COMET providing the vision 
for the programme, and the commitment to make it happen: 

COMET is passionate about the programme. No-one else at [tertiary 
partner] has an emotional involvement. 

COMET leads the school - they had a bigger vision than the school to 
start with. Their staff sells it with passion. 

All respondents were clear that an independent organisation was needed in the 
initial stages to get family literacy off the ground. They reported that COMET’S 
vision and enthusiasm led potential partners to be interested in the concept at 
the pilot stage, and that it was COMET that got the partners to the table. 

The leadership role was seen as significant because there were so many issues 
that were over and above the responsibilities and requirements of any one of the 
partners and also outside their natural area of work. 

Each partner has its own curriculum, different time allocations (semesters, 
terms, sessional day care), the holidays don’t synchronise. 

Their head is in family literacy all the time. We don’t have the time, it 
wouldn’t get off the ground. 
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Several respondents talked about the broader role COMET played in arguing for 
the continuity or growth of the programme. 

They have to keep on doing publicity to recruit and retain families and to 
retain support from government. 

COMET accessed funding streams. They have been invaluable in 
promoting the model. They have energy and resources; have got family 
literacy in front of ministers. 

A related role is brokering partnerships at the start of programmes. Respondents 
see this as a highly skilled and specialised role and it is known to take a large 
amount of time. 

COMET’S wider view and cross-site responsibility allows it to transfer solutions 
to issues that arise in one site to others. For example, at one school Internet 
access for the adult students was initially problematic and had flow-on 
implications for school IT policy, systems and on-going resources. This outcome 
then becomes something to be planned for on other sites. 



6.2 Aligning the components to maximise programme outcomes 

MFLP seeks to achieve outcomes for adults and children that are different from, 
and in addition to, those required of each partner individually. This cumulative 
effect is central to the concept of family literacy (Benseman, 2002) and requires 
a considerable degree of collaboration and innovation. In the eyes of 
respondents, COMET played a vital role in bringing about this complex 
alignment. 

The schools and early childhood centres are responsible for the learning 
outcomes related to children’s literacy. Similarly, the tertiary partner is 
accountable for the tertiary curriculum and the formal adult literacy learning 
outcomes for the adults (the qualification). 24 

Parent education and PACTT times result in learning outcomes for both adult 
participants and children. Schools and early childhood centres are not 
responsible to the learning outcomes of adults in any conventional sense. Some 
(but not all) aspects of these components may be related to the curricula and 
tertiary qualification requirements of the tertiary provider. For example, when the 
tertiary programme is linked to an early childhood educator qualification (as it 
was at Rowandale during this evaluation) elements of both may be able to be 
linked to the curriculum. However, at Bairds Otara, where the adults were 
enrolled in a variety of courses at Manukau Institute of Technology, it was more 
difficult to align PACTT and Parent Education to the study the parents were 
undertaking. 



24 The tertiary partner is required to report to COMET on the achievements of the adult 
participants, so COMET can advise the TEC as part of the funding contract. 



MFLP summative evaluation 2004 




- 78 - 



Both PACTT and Parent Education contribute to an outcome sought by the 
Ministry of Education - ‘engaging families in schools’ - which has become 
increasingly important to COMET and also more significant for schools and early 
childhood centres over recent years. Despite its importance to the Ministry of 
Education, this outcome is not a stated priority for the Tertiary Education 
Commission, which is the major tertiary education funder (and is therefore not 
funded by them). 

To maximise the benefits of MFLP, the partners have to work collaboratively, 
understanding each others’ curricula and finding opportunities for them to mesh. 
Collaboration across the different parts of the education sector is apparently not 
common - “There is no tradition of partners from different sectors in education 
working together.” Most respondents saw COMET as integral to that 
collaboration taking place. 

[COMET] gets the partners together behind the scenes. They make sure 
it all keeps working. 

COMET challenges everyone to think about their core business and 
different ways of delivering. 

Respondents said the best example of alignment working in practice thus far has 
been Stories together project at Rowandale. This project encompassed a range 
of activities. 

> All MFLP parents and children read the same book during PACTT over a 
number of weeks. The Strengthening School Libraries programme, run by the 
National Library, was already operating at Rowandale School and contributed 
1 0 copies of the same school library book so this could happen. 25 

> An author and an illustrator of children’s books were brought in by the 
Regional Coordinator to discuss story-writing and publication as part of the 
adult literacy programme. 

> The families made up books as part of their early childhood development 
curriculum and PACTT. 

> The parents made up a production of their own writing and presented it at 
Whanau PACT at the end of term. Respondents talked about the enthusiasm 
participants had about the production and how learners took a lot of the 
responsibility and initiative for it. 

Respondents thought this integrated project was very important and beneficial, 
although direct benefits were somewhat intangible. 



25 The National Library was delighted to contribute because it added value to their 
project by raising the profile of the National Library and by making connections between 
their work and the school and early childhood curriculum. It is also an example of how 
MFLP drew on the resources of another programme to enhance both MFLP and that 
other programme. 
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It’s hard to quantify the benefit - it got parents talking about the book; the 
author’s visit was inspiring. 

Another example of benefits gained from the partners aligning their programmes 
is the way Tahi PACTT time is changing each term at Rowandale so that adults 
and children work together on different elements in the curriculum (for example 
reading in one term, numeracy or oral language in another). This activity 
requires flexibility in the timetabling of the adult education component. 

To make the model more complex, COMET has to influence the programme 
rather than direct it, because all staff (other than those from COMET) are 
employed within their own sector. COMET’S view is that it takes some time and 
considerable help for the partners to come to an understanding about each 
other’s curricula, to develop professional respect and support for each other’s 
policies and then to work in depth to operationalise a collaborative programme. 



6.3 Recruiting and supporting learners and their families 

COMET staff said that both the recruitment and on-going support of families 
(including those families from previous years) was an on-going challenge. Lead 
teachers in the schools played a vital role in identifying families, but COMET was 
also heavily involved. Significant work is needed to recruit learners with no 
history of school or post-school educational success into a tertiary education 
programme and this role is labour intensive. Many of the adults are apprehensive 
and diffident about school and educational institutions generally. Respondents 
described COMET staff standing in school car parks before and after school at 
the start of school years or semesters, and visiting families identified by other 
people as possibly interested in the programme. They also did home visits as 
required. 

The adult educators reported that they got considerable support from COMET 
staff when major crises in their students’ lives meant they required extra 
assistance. COMET staff said their job was to identify barriers to learning and get 
them removed or minimised. This might mean for example going with a learner 
when their early childhood subsidy was cut by Work and Income or advocating 
when their social worker wanted them to leave the course and get a job. It also 
meant supporting the adult educator, while she assisted the students. 



6.4 Sourcing and accounting for funding 

COMET purchases educational programmes for adults from tertiary providers, 
either from the Adult Literacy Learning Pool or EFTs funding via the tertiary 
provider. COMET is accountable for this funding and both reports on it as 
required and supports the partners to meet any accountability requirements. 
COMET itself is not a tertiary provider, so meeting the reporting requirements of 
the funding system (geared as it is to providers) can be difficult. Funding to 
support the children’s literacy component is currently part of the adult literacy 
funding package. Schools receive a contribution from this funding towards the 
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use of their facilities for the adult education component, resources and teacher 
release time to allow for liaison and joint planning. 

Neither the Adult Literacy Learning Pool, nor the EFTs funding system (or the 
school funding system for that matter) can easily allow for a programme that 
seeks outcomes across diverse partners outside the scope of each funding 
source, and that seeks to both achieve and report on outcomes over and above 
what is normally required of each sector. 



6.5 Administration and support 

All respondents talked about COMET’S roles in administering the programme, 
taking the day-to-day load off the partners. The specific functions that were 
described included: 

> Attending weekly operational meetings at each site, involving the adult 
educator, the liaison teachers from both the school and kindergarten plus 
COMET; these people were in daily contact at the start of the programme 
as necessary. 

> Running regular management meetings with the designated decision- 
makers for the partners (i.e. a senior staff member from the relevant 
Kindergarten Association, the school and the kindergarten plus COMET). 
There was weekly contact (or even daily if necessary during recruitment 
and enrolment time) with a move out to fortnightly once the programmes 
were up and running. 

> Background administration - sending out agendas, keeping and 
distributing minutes of all the meetings etc. making PACTT-time happen. 

> Finding resources; both in terms of people and funding. 

> Organising Whanau PACTT - finding resources, networking with other 
organisations to build in special features into the curriculum; organising 
food. 

> Celebrating success. COMET was seen as very good at setting up events 
to acknowledge the progress participants were making. 

> Building networks with organisations in the community and between the 
sites. 

> Taking the load off partners - organising graduations, guest speakers. 

> COMET staff saw they had a staff development role, but the other 
respondents did not mention this. 



6.6 Crisis management 

The crises described by respondents ranged from not enough families being 
recruited as the deadline for funding approaches, personal crises for students 
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and resources and systems not being ready when required. COMET staff were 
seen as both a sounding board and also active solvers of problems. 

COMET identified childcare as a significant area of concern and this issue is 
discussed more fully in Section 6.3. 



6.7 Specialist expertise 

In general, respondents saw that COMET provided expertise that was outside 
their own sphere of interest, influence or role. The contacts the Regional MFLP 
Coordinator had with other educational sectors and the community were 
particularly valued. 

[The programmes] need skills, knowledge, contacts and time; schools and 
tertiary providers are fairly autonomous - early childhood centres are not. 
Its really important to know how schools and tertiary work. Schools don’t 
have time to find out. 

Schools don’t have networks outside their immediate school functions - 
they don’t have time. 

7 THE NEED FOR INDEPENDENT LEADERSHIP 

Respondents were asked if an independent organisation such as COMET would 
be needed if the family literacy concept were to be developed outside Manukau. 
All but two of the respondents believed the leadership and active involvement of 
an over-arching organisation like COMET was needed and the dissenting voices 
acknowledged that the independence and time COMET had was necessary 
during the pilot stage. 

Two main reasons were given for needing an independent broker in the project. 
The first was the complex understanding that was needed about all aspects of 
the education system (early childhood, schools and tertiary) which were beyond 
the scope of what each of the partners would normally be required to know. 

COMET understands each partner and their processes and systems on 
each site. Each site is different and run on different lines. 

Without COMET the programme would collapse. It needs an independent 
body to oversee everything; someone who knows what has to be done; 
who can look at each partner. They have to understand our systems. 

The second issue was the workload associated with the programme that was in 
addition to the core business of each of the partners. People accepted an 
increased workload within their jobs because they were committed to the 
programme; however, taking over the sorts of roles COMET undertook was seen 
as well beyond their scope and sphere of expertise. 
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We are not trained to do finances, to negotiate memoranda of 
understanding. 

We employ the principal to get the best education for our children. If we 
had to set up family literacy ourselves, we would have looked for 
someone to run it because the principal’s job is already too big. 

It’s harder than I ever imagined and takes more time because every 
partner needs to time to adapt their systems and ways of thinking and this 
is only one small part of their jobs. 

In addition, respondents mentioned the importance of COMET’S independence 
and the fact that they were therefore able to take a wider and more strategic 
perspective than the partners themselves, who were more influenced by the 
needs, conventions and systems within their own sectors. 

It’s hard to think outside the square when it’s your own sector or 
organisation that has to change. 

COMET is neutral and that’s pivotal to the role. They look at things that 
are best for learners and Manukau City, they’re not aligned to the 
providers - not captured. 

How would they deal with problems - ‘the rules don’t allow it’, without an 
external driving force. Practical things are problems, where structures 
work against families. 

If there was no COMET-employed coordinator, a few respondents said the only 
way the project could continue would be if they employed someone to act as 
one, because of the workload. However, it was pointed out that having one of 
the partners employ the person with overarching responsibility for making it all 
happen would upset the balance between the partners - it would be ‘owned’ by 
that partner, and would cease to be a collective of equals. 

Respondents did acknowledge that once sites got established the workload 
declined. Recruitment of students was a very busy time, but once that period 
passed and the programme was running for the year, things became more 
manageable. Stability of partners and the adult education programme would 
certainly also make a difference to the workload, as would more certainly of 
funding. As programmes matured and staff in them became more experienced, 
outside coordination would be less significant (until there was a crisis). 

8 CHALLENGES AND ISSUES DURING THE PILOT 

While every effort has been made to ensure anonymity in all other aspects of 
this evaluation, in the discussion of the most major issues faced by MFLP, the 
institutions have been named because the facts make them recognisable. 
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The most major issue involved the model of adult education to be used and the 
second, related issue, was childcare, to enable the chosen model to be put into 
place. 



8.1 The delivery of the adult education component 

At the start of the pilot, the intention was for the adult education component to be 
delivered on-site at Bairds Mainfreight School. MIT became the tertiary partner 
and the programme ran that way for two semesters. During that time, MIT 
identified the adult programme component that focused on parenting skills as 
too prescriptive and value-laden, and argued that the emphasis should change 
to child development and early child teaching and that shift took place in the 
curriculum. 

MIT then wanted the programme re-structured for the adults to be mainstreamed 
into MIT on-campus classes for 2004. MIT staff believed on-campus delivery 
would give the adults much more choice of programmes (they doubted that 
every student wanted to do the same programme) and would better meet their 
learning needs. MIT saw foundation education as ‘the crux’ for the cohort being 
sought by MFLP, and the specialist skills of the foundation learning team would 
not be available to students in the off-campus option of a specially designed 
course. Some students had lower literacy levels than others and needed more 
foundation learning support, others went on to participate in modules of teacher 
aide or social services courses and all of them got exposure to specialist 
foundation education maths teaching. 26 

It was the open-ended training that allows them to go on [to other tertiary 

study]. 

MIT is only a few hundred metres from Bairds Mainfreight School and MFLP 
students were able to move quite easily from their rooms in the school to MIT to 
attend classes and use the library and other facilities. The programme was 
altered so that some group time and teaching took place on the school site, but 
about two thirds of the adult programme shifted to the MIT site. 

This modus operandi for the programme was very different from the model 
COMET had envisaged and there appeared to have been vigorous 
disagreement. On the one hand, MIT argued that their prime concern was the 
educational progress of the adults. They believed the MIT-based option got the 
adults to progress much more effectively and that their on-campus proposal was 
educationally sound for the adults and viable in the long term. They saw real 
benefit in MFLP participants being mainstreamed with other students. 

If they get moved out and mix with other students (not like themselves) 

they see they can do it. 



26 Interestingly, more students from Bairds Otara perceived they had improved at maths 
compared to Rowandale and the degree of gain they felt was considerably higher. 
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COMET wanted someone on-site; MIT wanted to give the students 
exposure to the tertiary context. Students didn’t want to get out of their 
comfort zone, but within two weeks they were enjoying going to MIT and 
socialising with other students. 

They perceived COMET as inflexible in this respect, to be arguing for the 
programme rather than the participants and not allowing the model to evolve 
according to need. 

COMET wanted to keep the model pure compared to the operational 
realities of this provider. COMET had a clear picture of what is should 
look like, rather than letting it grow and develop organically. 

On the other hand, COMET and the school thought that while adults benefited, 
the programme as a whole did not, because the focus on whanau that is integral 
to it was reduced. Having the adults off-site and participating in a range of 
different courses broke up the integrity of the model and the overall cohesion of 
the group. As far as they were concerned, Bairds Otara MFLP then ran in a way 
that “was good for MIT and good for the adults, but less beneficial for the 
programme.” They thought MIT did not recognise that MFLP was a programme 
with a number of components of equal value but treated it simply as an adult 
education course they were offering off site. 

It was better for adult ed [and MIT], For PACTT and Parent Education it 
was not better. They [the parents] struggle because they are not on-site 
as often - it is more disjointed, and harder to integrate Parent Ed/PACT. 
The aspects that are about families supporting their children’s learning 
suffered. 

MIT did not think the other partners were prepared to understand enough of the 
timetabling and resource realities under which the tertiary provider had to 
operate. 

They wanted a course at school, flexible and one that was to be dropped 
at any time to suit the demands of the school timetable. 

However, whereas MIT thought these points were limitations (flexibility to fit in 
with the school timetable for example), COMET regarded them as important 
elements for creating an integrated programme. For example, when the school 
went on trips, MFLP participants took part, both to make a contribution back to 
the school for all the support they had been given, and also because being 
involved in the school-based life of their children was as important to the 
programme as the adult education component. 

COMET were also concerned that changing the adult education model at Bairds 
Otara during the pilot to one of off-site delivery was unrealistic, given that it 
wouldn’t be feasible at other sites where it would be unlikely the school and 
tertiary provider were across the road from each other. 

The issues that emerged from this disagreement were sufficiently resolved for 
the programme to run during 2004, but the resulting philosophical disagreement 
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has remained. For 2005 COMET has sought a partner who is prepared to deliver 
an on-site programme. Both AUT and the Auckland College of Education 
participated as tertiary partners offering on-site programmes during 2004 and 
AUT is going to be the partner on all sites for 2005, offering an on-site certificate 
level course - the Certificate in Introduction to Early Childhood Education. This 
arrangement will offer an opportunity for the adult educators to share 
experiences and for the programme to bed in with one tertiary provider. 

Section 9.5 discusses the adult education component of MFLP more generally. 

8.2 Childcare 

As mentioned earlier, COMET staff perceived that overall, childcare was one of 
the most major issues that MFLP had to deal with. The programme was not able 
to systematically meet the childcare needs of parents with children aged under 
two years, which restricts the ability of the programme to recruit participating 
families. 

The other major and on-going childcare issue arose from the way the adult 
education component was to be delivered. When the adults on the Bairds Otara 
programme mainstreamed into MIT, some of their classes began at 8 am and 
ran at different times to the school/kindergarten timetable. Bairds Kindergarten, 
the early childhood partner, was licensed for two sessions from 8.30 -3pm, so 
there was no childcare available at the start of the day for parents to attend early 
classes. Also, some MIT classes ran into lunchtime when there needed to be 
childcare cover as well, to fill the gap between the two childcare sessions. This 
had not been an issue when parents were on-site because they had their young 
children with them over lunch. MFLP eventually resolved this issue by funding a 
teacher aide to cover at the start and middle of the day. However, identifying the 
issue and trying to find solutions was apparently very stressful. 



8.3 Meetings 

The concern expressed by the most respondents was the number of meetings 
the project required. People understood that collaborative decision-making was 
part of the programme’s philosophy and recognised its importance. 

I was challenged by others about the amount of time spent in meetings 
but all partners have to understand, always work to be done like when 
you are negotiating PACTT, you have to do it with the kindergarten as 
well as the school. 

But a number of respondents were frustrated by the number of meetings, the 
fact that key people were late so meetings did not start on time, meeting process 
was “woolly”, sometimes they did not have clear agendas and were not 
sufficiently focused. The tertiary partners in particular thought COMET was 
‘time-rich’ (unlike themselves) and somewhat unrealistic about time demands 
being made on behalf of a relatively small number of participants. One 
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respondent said the project was “over-managed”. However, other respondents 
were complimentary about the amount of work that was achieved at meetings. 

A number of respondents referred to frustrating meetings where decisions could 
not be made because the MFLP Regional Coordinator did not attend (although 
the new project coordinator came in her place, sometimes she could not make 
decisions either). A number said the Regional Coordinator was increasingly hard 
to get hold of, as the number of sites grew. They did acknowledge that she was 
good at getting back to them on issues by email or phone. It was also 
acknowledged that the need for meetings declined once programmes became 
more established and staff became more experienced - i.e. they reduced once 
students were recruited and enrolled, and again later in the year once 
programmes were established. 

Several respondents (including, but not only, tertiary partners) talked about the 
amount of time the programme required being disproportionate for only a small 
number of participants. However, some also weighed that up against the 
benefits, which they say outweighed the effort. One person said she had been 
cynical at the beginning because of the work demands for only nine students, 
but now could see how worthwhile it had been. 

I saw differences on an everyday level, not just for those children [in 
MFLP] but other children in kindergarten. It had a flow-on effect to other 
children and kindergarten community. Some times it felt like it required a 
huge organisation. But it was like ripples in a pond, going out and out and 

out I could see that something was happening. I learned how powerful 

parents can be. I always knew it but something like this can make them 
even more powerful. 

In a few cases, respondents talked about communication problems - mail sent 
to wrong addresses, notices about meetings that did not happen etc. However, 
these difficulties did not seem to be out of keeping with what might be expected 
of a pilot programme. A small number of people working at operational level 
talked about needing to know what was going on at the strategic level, so they 
could better understand why funding wasn’t available for particular projects or 
ideas or to know what the problems were. 



8.4 Aspects of funding and administration 

COMET has to apply for funds and report on provision arrangements and 
outcomes for programmes that are not readily comparable to others. For 
example, the Adult Literacy Learning Pool tends to fund short courses or a staff 
position on one site or in one institution. The fact that COMET funding is for a 
full-time course running (in different ways) on more than one site is not easily 
accommodated. The diverse learning outcomes achieved by adults and their 
whanau, schools and also the wider community make it difficult both to get 
funding and report on the programme in a holistic way. 
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9 EVOLUTION OF THE MODEL 

The pilot of family literacy programmes in Manukau has now been running for 
two years, involving six intakes of students from three schools, two early 
childhood centres and three tertiary providers. In the light of this depth of 
experience, while it was not part of the brief for this report, we think it is timely to 
reflect on the model and how it has developed. Our involvement with COMET 
and family literacy and goes back to its inception and John Benseman has 
carried out three evaluations of MFLP, so we are well-placed to make some 
comments. Our comments and questions are interwoven with observations 
made by key informants during the interviews reported on earlier. 

The issues we discuss here will not be exhaustive and solutions to the questions 
raised will only evolve over time and through discussion with all the 
stakeholders. Nor should they be seen to detract in any way from the overall 
quality and benefits gained from the MFLP to date. 



9.1 C reating a unique model of family literacy 

The MFLP approach is innovative and challenges the partners to think outside 
the square, which most partners have found to be rewarding and to have added 
value to their institution and to the families involved. Plowever, the very nature of 
that innovation makes it more complex to fund and to organise and requires 
extra effort in every sphere. 

Initially, a great deal of effort had to go into developing a model that took into 
account the significantly different conditions here compared to those that have 
grown up around the Kenan model in the US. For example, in the USA, funding 
is available for all four components of the model from one source - a ‘one-stop- 
shop approach. It does not have to be cobbled together from a variety of 
sources as it does here. 

Also, in the US, the early childhood and adult education curricula are designed 
and delivered by specialist staff directly employed by the family literacy 
programme. Therefore, they do not have to develop unique partnerships and 
either use or develop specific curricula per programme as has had to happen 
here. Additionally, US family literacy programmes have support services built 
into the programme. Typically, there would be one coordinator per eight 
programmes, plus that cluster of programmes also would have a social worker 
and career counsellor to help the adults move on to other study or work. 

A funding regime more appropriate to the programme that could be replicated 
between sites would reduce the time required to organise and broker partnership 
arrangements, time that would be freed up to work on other programme 
development issues. 

Developments such as Tahi PACTT and Whanau PACTT show how the 
programme is evolving to fit the Manukau (and national) context and to reflect 
Maori and Pasifika values. Plowever, comments from respondents also suggest 
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these aspects of the programme involve a great deal of time and many 
meetings. Do the benefits and learning outcomes from whanau and extended 
Whanau PACTT warrant the amount of planning and energy involved in making 
them happen? How could this be made more manageable? 

The sites appear fairly autonomous. As the programme matures, how could 
more links be built between sites and more be learned by cross-fertilisation of 
ideas and experiences? Is it appropriate or important there should be, given the 
different characters of the programmes on each site? Has that difference come 
about more because of the adult education programme than any other element 
and if so, does it matter? 

One assumes that economies of scale can be achieved from having a number of 
programmes running. As programmes mature, more of the regional coordinator’s 
time will be spent on new programmes and longer-running programmes will 
carry more responsibility. What will be a realistic workload for the regional 
coordinator? 



9.2 Reviewing the MFLP pilot and strategic planning 

As yet, there has not been a detailed de-brief between the partners in the pilots 
and other key stakeholders (including operational and management staff) to both 
finish off this phase of the project and try to bring together the detailed insights 
each sector has gained. This short section of the report might prove a starting 
point for that discussion. 

A review process could usefully start with reviewing all aspects of the model to 
consider their strengths, applicability and limitations, in order to see how they 
might evolve. Such a review could end with the creation of cohesive MFLP goals 
that all programmes and partners clearly identified with. 



9.3 Sustainability 

There is some concern about the sustainability of the model within an individual 
school. Assuming in a typical school there are 100-150 families with four 
children each, only 50% of those families might meet the family literacy project 
criteria. Twenty families go through in two years. What is a realistic number to 
assume will want to go through, and how many years would it take before the 
programme was no longer viable? Could the programme be owned by a cluster 
of schools, or a community rather than be based in an individual school, to 
increase the potential catchment of families? For example, the Bairds 
Intermediate site might be a physical site for a programme that drew more 
widely across the community. If it were to move out of an individual school, the 
logistics of getting parents back to their respective schools for PACTT time etc. 
would need to be funded and allowed for. 

Sustainability is also related to funding. At present the programme runs year by 
year and because the funding of the coordination by COMET is more difficult to 
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access (but no less important) than the other components, it is the last to be 
confirmed. Indeed, all funding was confirmed very late in 2004. This delay 
impacts most on the ability of providers to ensure they have an appropriately 
trained adult educator in place. Respondents talked about the frustrations of 
never knowing until the last minute whether the programmes could get started, 
the difficulties of recruiting and retaining highly qualified staff for year-to-year 
contracts and the time pressure that results when funding decisions are made at 
the last minute. The lack of surety was seen as a significant limitation in the 
current model. 

A practical issue related to sustainability is whether appropriate physical space 
can guaranteed for adults if a school roll grows. Would a school be required to 
take back the classroom(s) used by family literacy because the property/school 
roll formula would not be able to take its use by adults into account? 



9.4 The staffing ratios and costs involved 

The tertiary partners argued strongly that they do not make money from their 
courses; what is less clear is whether they cover their costs and by how much. 
COMET has not been able to establish clearly how much EFTs funding is 
generated by the courses. Having to deliver a specially designed programme (as 
was used in the beginning) is more expensive than using a programme already 
in existence that can be adapted for off-campus delivery (in the way that AUT is 
taking a pre-existing six month-long certificate programme and delivering it off- 
site over one year). 

The 1:10 staff student ratio used is below normal tertiary class ratios of 1:17, 
which make them expensive. However, this low ratio is important because it 
helps to build a cohesive group, which is seen to be important for ensuring 
success for the adults. It is also realistic because although the adult is the one 
enrolled in the adult component, the actual participant ‘unit’ is a family, not an 
individual. The adult educator (as well as all other staff) is working with not just 
10 adult MFLP participants, but also the ten MFLP-nominated children, plus 
other children in those families and wider whanau members as well. 

At least one of the tertiary partners had allowed for .2 staffing for the liaison staff 
member involved, taking into account the meetings and additional work required 
because the programme is seen to be management-intensive. Some of that time 
demand is because there is additional work for the tertiary provider in supporting 
a staff member off-site, who is working in an isolated position teaching students 
with complex educational and social needs. The model has built into it some 
funding for the partners to acknowledge liaison time, but it would be timely for 
consider now to whether that funding actually matches what actually takes 
place. 

Greater transparency of tertiary course costs will be needed if the model is to be 
promoted for use in other areas. Schools and early childhood centres are not 
required to generate revenue from their activities in the same way and did not 
identify the cost of the programme to them as an issue during the interviews. 
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9.5 The adult education component 

The comments made by respondents clearly identify the adult education 
component of the programme as the most challenging, because it is where 
adult-related funding and curricula interface with a primary school site, ECE and 
school curricula and learning outcomes that are outside the norm for all parties. 

In the life of the pilot, there has been three ways of providing that component. 
First, a single tertiary provider offered a specialised course on site. Then, three 
providers offered three different qualifications (one of which was partly off-site 
and where students took part in a range of programmes). Now, during 2005 one 
provider (AUT) will offer one course on three different sites. 

As these different organisational models are explored, more fundamental 
questions are still to be debated fully by the partners and stakeholders. 

What should the features be of the adult education component? Presumably, it 
has to be sufficiently generic programme so it can be delivered in a number of 
contexts, without constant re-organisation of structure and funding. A nationally- 
applicable model may lack flexibility, but make up for it by enabling systems and 
procedures to be developed that minimise the workload when a new programme 
site is being developed. On the other hand, a site-specific programme could 
adapt to local need e.g. if the largest local employer in an area was a hospital, 
perhaps an introductory heath care qualification would be more appropriate than 
a child development-oriented programme. However, this approach would 
demand greater coordination as the requirements on each site appear to be 
heavily influenced by the flexibility possible within each adult education 
qualification. 

Secondly, it has to able to integrate into the other components of MFLP (in 
particular child development, which is why introductory early childhood teaching 
qualifications are useful). Thirdly, it has to be broad enough to meet the needs of 
diverse student groups on different sites. Finally, the findings of the evaluation 
suggest it is important that the adult component deliver a qualification to the 
participants to provide a pathway through to higher levels of study or work. 

Meeting all these requirements is challenging and the pilot may not yet have 
come up with the right course mix. Perhaps a purpose-designed family literacy 
programme would be more effective, incorporating generic and some optional 
elements that would allow sites some flexibility. An evaluation at the end of 
2005, after running one programme on three sites will provide some useful 
insights. Also, existing options such as the National Certificate of Employment 
Skills could be reviewed to see if they could be flexible enough to meet the 
programme needs. 

A related issue is the ability of MFLP to meet the needs of any adults with very 
low literacy or ESOL needs. In the earliest intakes, people with very low literacy 
levels were referred on, rather than included. Specialist skills are needed to 
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teach adults with foundation learning needs (i.e. ESOL or literacy), yet the job 
descriptions for the adult educators are directed by course content and appear 
to have less emphasis than expected on skills related to the teaching of literacy 
and language to adults. How can the programme ensure that there is the 
capability to work with learners with high literacy/foundation skills needs? 

If MFLP is to expand within Manukau, MIT and COMET need to debate the adult 
educational issues more comprehensively (perhaps as part of a broader forum), 
in order to MFLP to be able to utilise MIT’s location in Manukau and its strengths 
as one of the largest providers of foundation education in the country. 



9.6 Issues for learners 

Some parents may have trouble making the time needed available for a full-time 
course. Many people engaged in foundation learning-related study do so part- 
time and many of the programme participants work part-time (a few actually 
worked full-time at night and were considered full-time students by day). In the 
USA, there are programmes organised for part-time students. This option would 
be challenging but worth investigating. 

Staff have had to work very hard to recruit families at every intake; COMET and 
school staff stand in school car-parks, visit families, follow up from suggestions 
from other students etc. Part-time programmes may make that easier and may 
increase the opportunity to recruit men (there have only been four men so far 
across the programme). Adult education content geared to childcare is not likely 
to be particularly attractive to men. 

Currently the programme cannot cater for children under three and childcare 
issues have occurred across all sites at times. MFLP has also concentrated on 
adults with primary and pre-schoolers. A new programme involving older 
children is being planned for later in 2005. This development will raise a new set 
of issues, but there is no doubt that some of these (in particular, childcare/after- 
school care) can be more readily dealt with if there is appropriate funding. At the 
end of 2005, the experiences of a programme based on older-children and their 
parents need to be recorded as part of the end of year evaluation. 

COMET has had contact from a number of iwi, who are interested in adopting 
the model. Will the model work as it stands if it was targeted to one group of 
learners (for example either Maori or Pasifika)? What would be some of the 
issues, from MFLP’s experience if another community wanted to try that 
approach? 



9.7 G reater recognition of the work involved 

The time involved to establish and participate in on-going management of the 
programme is not yet fully recognised in the job descriptions of some of the key 
participants (other than COMET). 
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The adult educator’s job effectively has two parts - teaching the programme and 
‘facilitating a learning community’. The adult educator has contact on a daily, or 
at least weekly, basis with adult students, their children, the school and early 
childhood staff. The adult educator has to relate to all of those people, and to 
understand the components of the programme on a day-to-day basis within the 
school and the early childhood centre, in order to keep the adult programme 
running and integrated with the others. It is not clear that the job descriptions for 
this position consider this complexity. In addition, the degree of personal support 
and guidance that the adult educator is required to give to students when 
operating alone in a school may not be adequately recognised within their 
workload. It is unrealistic to say those support services are available from the 
tertiary provider because it is highly unlikely students will go to a campus to 
which they are not connected. 

The new initiative, Second Step, where students are supported after they leave 
MFLP and move onto further study, has yet to be evaluated. Is it needed and 
who best should offer it? If it is to be expected as part of the work of an on-site 
adult educator, how is it to be incorporated into an already very full role? 

Early childhood, school and tertiary respondents all mentioned that in addition to 
the meetings they attend as part of their jobs, being involved in family literacy 
has involved work outside work time. They become involved in the life of the 
programme and the families and attend graduations, Whanau PACTT, powhiri to 
welcome new groups etc. While people have chosen to go the extra mile, it is 
something that needs to be taken into account when planning events and 
looking for support for additional activities. 

10 CONCLUDING REMARKS 

The findings of this section of the evaluation are completely in keeping with the 
findings of the other evaluations of MFLP over the last two years - the 
programme has offered a great deal of benefit to the individual adult students 
and their families and to the schools and early childhood centres that have taken 
part. 

There was great good will toward COMET as the instigator of the concept and 
recognition of the skill, knowledge, energy and commitment COMET staff have 
brought to the role of coordinating the programme. 

The issues that have been raised in this section are typical of the nature and 
purpose of a pilot programme - to see if something will work and the factors that 
impact on success. The goodwill of the partners has meant they have been able 
to work through issues as they arise for the benefit of the participants. 
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APPENDIX A - MFLP GOALS FROM 2003 WORKSHOP 



Bairds Otara 



Goals 


Adults 


Children 


Increase literacy skills 


X 


X 


Improve attitudes to reading and learning generally 


X 


X 


Increase the use of books 


X 


X 


Increase and improve reading to and with children 


X 




Learn how to learn 


X 




Improve self-confidence 


X 


X 


Increase educational aspirations 


X 




Increase awareness of opportunities - education and work 


X 




Spend more focussed/special time between children and 
parents 


X 


X 


Demonstrate greater awareness of parental roles and 
responsibilities 


X 




Demonstrate greater encouragement of children 


X 




Participate more in school/kindergarten activities 


X 




Increase contact with child’s teacher(s) 


X 




Maintain attendance 


X 


X 


Improve transition to school 


X 


X 


Increase parent/child outings together 


X 


X 


Improve links between school and kindergarten 






Increase and improve child/parent interaction 


X 


X 


Increase parents’ awareness of hauora 


X 




Show commitment to on-going learning 


X 


X 



Rowandale 



Goals 


Adults 


Children 


Increase literacy skills 


X 


X 


Improve attitudes to reading and learning generally 


X 


X 


Increase use of books 


X 


X 


Increase and improve reading to and with children 


X 


X 


Increase vocabulary 


X 


X 
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Reduce transiency 


X 




Improve links between school/kindergarten 






Increase parent/child interaction 


X 


X 


Increase parents’ awareness of hauora 


X 




Increase self-confidence 


X 


X 


Improve attitudes towards school 


X 


X 


Increase & improve help with homework 


X 




Increase ECE attendance to min. of 1 yr 




X 


Maintain attendance 


X 


X 


Increase use of outdoors for educational purposes 


X 




Increase use of te reo 


X 


X 
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